ElixirScope: AI-Powered Execution Cinema Debugger for Elixir
ElixirScope is a next-generation debugging and observability platform for Elixir applications, designed to provide an "Execution Cinema" experience through deep, compile-time AST instrumentation guided by AI-powered analysis.
π¬ Try the Cinema Demo Now!
Experience ElixirScope in action with our comprehensive working demo:
git clone https://github.com/your-org/elixir_scope.git
cd elixir_scope/test_apps/cinema_demo
./run_showcase.sh
β¨ See all features working in ~1 minute: Time-travel debugging, performance monitoring, state reconstruction, and more!
[π Complete Demo Documentation β](https://github.com/nshkrdotcom/ElixirScope/blob/main/test_apps/cinema_demo/FULLY_BLOWN.md
π― Vision & Mission
Transform Elixir development by providing unprecedented insight into application behavior through:
- Total Behavioral Recall: Capture comprehensive execution history through compile-time instrumentation
- AI-Driven Guidance: Intelligent instrumentation planning and code analysis
- Compile-Time Granularity: Access local variables, trace expressions, and inject custom logic with precision
- Time-Travel Debugging: Navigate through your application's execution history (planned)
- Multi-Dimensional Analysis: Correlate events across time, processes, state, and performance
π Current Implementation Status
ElixirScope has achieved major milestones with a comprehensive working demo and core components fully operational:
π¬ NEW: Complete Cinema Demo Available!
β FULLY WORKING SHOWCASE - Experience all ElixirScope features in action:
cd test_apps/cinema_demo
./run_showcase.sh
What you'll see:
- β All 6 demo scenarios running (Task Management, Data Processing, Complex Operations, Error Handling, Performance Analysis, Time Travel Debugging)
- β Real-time performance metrics and system monitoring
- β TemporalBridge integration with working time-travel debugging
- β Complete end-to-end workflow demonstration
- β ~1 minute comprehensive feature showcase
π See FULLY_BLOWN.md for complete demo documentation
β Production-Ready Components
Core Infrastructure (100% Complete)
- β
Configuration System -
ElixirScope.Config
with environment support and validation - β
Event System -
ElixirScope.Events
with comprehensive event type definitions - β
Utility Functions -
ElixirScope.Utils
with ID generation, timestamps, and data formatting - β Application Supervision - Robust OTP application structure
Data Capture Pipeline (95% Complete)
- β
InstrumentationRuntime - High-performance runtime event capture API (920 lines)
- Function entry/exit tracking with correlation IDs
- Local variable snapshots and expression tracing
- AST-correlated event reporting (enhanced for hybrid architecture)
- Phoenix, LiveView, Ecto, and GenServer integration functions
- <1Β΅s overhead when disabled, <500ns when enabled
- β Ring Buffer - Lock-free, high-performance event storage (supporting >100k events/sec)
- β Ingestor - Ultra-fast event ingestion (<1Β΅s per event target)
- β Pipeline Manager - Coordinated async processing with backpressure
- β Data Access - ETS-based storage with efficient querying
AST Transformation Engine (90% Complete)
- β Core Transformer - Production-ready AST instrumentation for functions and GenServer callbacks
- β Enhanced Transformer - Granular instrumentation with local variable capture and expression tracing
- β Injector Helpers - Comprehensive AST manipulation utilities
- Pattern Support: GenServer, Phoenix Controller, LiveView, and Channel instrumentation
AI Analysis Framework (85% Complete)
- β
LLM Integration - Complete multi-provider support:
- Google Gemini API - Production integration with authentication
- Vertex AI - Service account authentication and error handling
- Mock Provider - Comprehensive testing infrastructure
- β AI Orchestrator - Centralized AI coordination and instrumentation planning
- β Pattern Recognizer - OTP pattern detection (GenServer, Supervisor, Phoenix patterns)
- π§ Code Analyzer - Basic analysis with room for enhancement
- π§ Predictive Execution - ML-based execution path and resource prediction
Cinema Debugger System (90% Complete)
- β TemporalStorage - High-performance temporal event storage with AST correlation
- β TemporalBridge - Real-time bridge between runtime events and temporal storage
- β State Reconstruction - Working time-travel debugging via TemporalBridge
- β Event Correlation - AST node correlation with runtime events
- β Performance Tracking - Real-time metrics and system monitoring
AST Repository System (85% Complete)
- β Repository Core - Central AST storage with runtime correlation
- β Function Data - Comprehensive function-level analysis structure
- β Runtime Correlator - Bridge between AST nodes and runtime events (working)
- π§ Parser Integration - AST parsing with instrumentation point mapping (partial)
Compile-Time Orchestration (80% Complete)
- β Orchestrator - Complete instrumentation planning and AI integration
- β Configuration Management - Runtime configuration updates
- π§ Mix Task Integration - Automatic compile-time transformation (planned)
π Planned Features
Web Interface (Next Priority)
- Phoenix web interface for trace visualization
- Visual time-travel debugging interface
- Interactive execution timeline
- Multi-dimensional event correlation
Advanced Integrations (Future)
- IDE integration (ElixirLS compatibility)
- Distributed tracing capabilities
- Advanced AI analysis and recommendations
- Hypothesis testing framework
ποΈ Architecture Overview
ElixirScope follows a compile-time first approach with runtime correlation capabilities:
graph TD
subgraph "Developer Tools"
IDE["IDE / IEx"]
MixTask["Mix Tasks (Planned)"]
end
subgraph "AI Analysis Engine"
Orchestrator["AI.Orchestrator β
"]
CodeAnalyzer["Code Analyzer β
"]
PatternRecognizer["Pattern Recognizer β
"]
LLMProviders["LLM Providers β
<br/>Gemini β’ Vertex β’ Mock"]
end
subgraph "AST Transformation"
Transformer["AST.Transformer β
"]
EnhancedTransformer["Enhanced Transformer β
"]
CompileTime["Compile-Time Orchestrator π§"]
end
subgraph "Data Capture Pipeline"
Runtime["InstrumentationRuntime β
"]
Ingestor["Ingestor β
"]
RingBuffer["Ring Buffer β
"]
Pipeline["Pipeline Manager β
"]
Storage["Data Access β
"]
end
subgraph "Cinema Debugger System"
TemporalStorage["TemporalStorage β
"]
TemporalBridge["TemporalBridge β
"]
StateReconstruction["State Reconstruction β
"]
end
subgraph "AST Repository"
Repository["Repository Core β
"]
FunctionData["Function Data β
"]
Correlator["Runtime Correlator β
"]
end
IDE --> Orchestrator
Orchestrator --> CodeAnalyzer
CodeAnalyzer --> LLMProviders
CodeAnalyzer --> Transformer
Transformer --> Runtime
Runtime --> Ingestor
Ingestor --> RingBuffer
RingBuffer --> Pipeline
Pipeline --> Storage
Storage --> TemporalStorage
TemporalStorage --> TemporalBridge
TemporalBridge --> StateReconstruction
Repository --> Correlator
Correlator --> TemporalBridge
style Runtime fill:#c8e6c9,color:#000
style Ingestor fill:#c8e6c9,color:#000
style Orchestrator fill:#e1f5fe,color:#000
style Repository fill:#fff3e0,color:#000
style TemporalBridge fill:#c8e6c9,color:#000
style StateReconstruction fill:#c8e6c9,color:#000
Key Architectural Principles
- Compile-Time First: Primary focus on AST transformation for deep instrumentation
- AI-Guided Intelligence: Smart instrumentation planning based on code analysis
- High-Performance Pipeline: Sub-microsecond event processing with async correlation
- Production-Safe: Configurable instrumentation levels with minimal overhead
- Extensible: Plugin architecture for new analysis types and integrations
π Getting Started
Installation
Add ElixirScope to your mix.exs
:
def deps do
[
{:elixir_scope, "~> 0.1.0"}
]
end
Basic Configuration
# config/config.exs
config :elixir_scope,
# Data capture settings
capture: [
buffer_size: 10_000,
batch_size: 100,
flush_interval: 1_000
],
# AI analysis settings
ai: [
llm_provider: :mock, # :gemini, :vertex, :mock
analysis_timeout: 30_000,
planning: [
default_strategy: :balanced
]
],
# AST instrumentation settings
ast: [
default_instrumentation_level: :function_boundaries,
performance_monitoring: true,
variable_capture: false
]
Quick Start - Try the Cinema Demo!
The fastest way to experience ElixirScope is through our comprehensive demo:
# Clone the repository
git clone https://github.com/nshkrdotcom/ElixirScope.git elixir_scope
cd elixir_scope
# Run the complete showcase
cd test_apps/cinema_demo
./run_showcase.sh
This will demonstrate:
- β Task Management - GenServer state tracking and lifecycle management
- β Data Processing - Complex data transformation pipelines
- β Time Travel Debugging - State reconstruction at any point in time
- β Performance Analysis - Real-time metrics and bottleneck identification
- β Error Handling - Graceful recovery and error correlation
- β Complex Operations - Deep call stacks and recursive function analysis
Current Usage Examples
Main Application API
# Start ElixirScope with your application
children = [
# Your application children...
{ElixirScope, []}
]
# Use the main API
ElixirScope.start(strategy: :full_trace, sampling_rate: 1.0)
status = ElixirScope.status()
# Check if running
ElixirScope.running?()
# => true
# Update configuration at runtime
ElixirScope.update_config([:ai, :planning, :sampling_rate], 0.5)
Cinema Debugger Features
# Start TemporalBridge for time-travel debugging
{:ok, bridge} = ElixirScope.Capture.TemporalBridge.start_link(
name: :my_bridge,
buffer_size: 1000
)
# Get bridge statistics
{:ok, stats} = ElixirScope.Capture.TemporalBridge.get_stats(bridge)
# Reconstruct state at specific timestamp
{:ok, past_state} = ElixirScope.Capture.TemporalBridge.reconstruct_state_at(
bridge,
timestamp
)
AI-Powered Code Analysis
# Analyze code with AI
{:ok, analysis} = ElixirScope.AI.CodeAnalyzer.analyze_module(MyModule)
# Generate instrumentation plan
{:ok, plan} = ElixirScope.AI.Orchestrator.plan_for_module(source_code)
# Recognize patterns
patterns = ElixirScope.AI.PatternRecognizer.extract_patterns(ast)
LLM Provider Setup
Google Gemini API
export GOOGLE_API_KEY="your-gemini-api-key"
config :elixir_scope, ai: [llm_provider: :gemini]
Vertex AI
export VERTEX_JSON_FILE="/path/to/service-account.json"
config :elixir_scope, ai: [llm_provider: :vertex]
π§ͺ Testing & Development
ElixirScope has comprehensive testing with 723 tests passing, 0 failures and extensive coverage across all components.
Test Commands
# Default test command - excludes live API tests, shows full test names
mix test
# All tests including live API tests (may hang without API keys)
mix test.all
# Fast tests (parallelized)
mix test.fast
# LLM provider tests
mix test.mock # Mock provider only
mix test.gemini # Live Gemini API tests
mix test.vertex # Live Vertex AI tests
mix test.live # All live API tests
# Legacy alias (same as default mix test)
mix test.trace
# Performance validation
mix test.performance
Cinema Demo Testing
# Test the complete demo application
cd test_apps/cinema_demo
mix test
# Run individual demo scenarios
mix run -e "CinemaDemo.run_task_management_demo()"
mix run -e "CinemaDemo.run_time_travel_debugging_demo()"
# Run the complete showcase
./run_showcase.sh
Live API Testing
To run tests against real LLM APIs:
# Set up credentials
export GOOGLE_API_KEY="your-key"
export VERTEX_JSON_FILE="/path/to/service-account.json"
# Run live tests
mix test.live
π Performance Characteristics
Current Benchmarks
Component | Performance | Status |
---|---|---|
InstrumentationRuntime | <100Β΅s per event | β Production Ready |
TemporalBridge | <100Β΅s event forwarding | β Production Ready |
Ring Buffer | >100k events/sec throughput | β Production Ready |
Event Ingestor | <1Β΅s per event processing | β Production Ready |
State Reconstruction | <10ms for typical GenServer | β Production Ready |
AST Transformation | <100ms for medium modules | β Production Ready |
AI Analysis | <30s for project analysis | β Functional |
Cinema Demo | ~1 minute complete showcase | β Fully Working |
Memory Usage
- Base overhead: ~10MB for core system
- TemporalBridge: ~5MB per bridge instance
- Per-module overhead: 50KB (small), 200KB (medium), 1MB (large)
- Ring buffer: Configurable, 1-10MB typical
- Cinema Demo: ~15MB total during showcase
- Total overhead: <50MB for typical projects
πΊοΈ Development Roadmap
β Phase 1: Foundation Complete (May 2025)
Major Achievements
- β Complete Cinema Demo with all 6 scenarios working
- β TemporalBridge integration for time-travel debugging
- β State reconstruction capabilities
- β Performance monitoring and metrics
- β AST-runtime correlation system
- β Comprehensive test suite (723 tests passing)
- β Production-ready core components
Phase 2: Web Interface & Enhanced Features (Q2 2025)
Immediate Priorities
- [ ] Phoenix web interface for Cinema Debugger
- [ ] Visual time-travel debugging interface
- [ ] Interactive execution timeline
- [ ] Real-time event streaming dashboard
Enhanced Features
- [ ] Mix task for automatic instrumentation
- [ ] IEx helpers for debugging workflow
- [ ] Enhanced AI orchestration and planning
- [ ] User-friendly instrumentation API
Phase 3: Advanced Features (Q2-Q3 2025)
- [ ] Distributed tracing across nodes
- [ ] Advanced AI analysis and recommendations
- [ ] IDE integration (ElixirLS)
- [ ] Production deployment patterns
- [ ] Performance optimization tools
π€ Contributing
We welcome contributions! ElixirScope is built with a solid foundation and clear architecture.
Current Contribution Opportunities
- Web Interface - Build the Phoenix-based Cinema Debugger web UI
- Visual Debugging - Create interactive time-travel debugging interface
- AI Enhancement - Improve code analysis and pattern recognition
- Performance Optimization - Enhance event processing and storage
- Documentation - Improve guides and examples
- Testing - Add property-based and chaos testing
Getting Started with Development
- Fork the repository
- Set up development environment:
mix deps.get mix test # Ensure all tests pass (723 tests, excludes live API tests)
- Try the Cinema Demo:
cd test_apps/cinema_demo ./run_showcase.sh
- Create a feature branch
- Add comprehensive tests for new functionality
- Submit a pull request
Development Guidelines
- Test-driven development: All new features need tests (current: 723 tests passing)
- Performance-conscious: Maintain <100Β΅s instrumentation overhead
- Backward compatibility: Don't break existing APIs
- Documentation: Update guides for new features
- Demo integration: Consider adding features to Cinema Demo showcase
π License
ElixirScope is released under the MIT License. See the LICENSE file for details.
π Acknowledgments
ElixirScope builds on the excellent foundation of the Elixir and OTP ecosystem. Special thanks to:
- The Elixir core team for creating such an introspectable language
- The Phoenix team for demonstrating excellent debugging tools
- The Erlang/OTP team for the robust foundation
- The Elixir community for inspiration and feedback
π Support & Community
- Documentation: [Coming Soon]
- Issues: GitHub Issues
- Discussions: GitHub Discussions
- Elixir Forum: Tag your posts with
elixir-scope
ElixirScope: Transforming Elixir development through intelligent instrumentation and AI-powered analysis. The future of debugging is here - and it's working! π¬β¨