Guide: The Signal Bus
View SourceThe Signal Bus is the heart of Jido.Signal's pub/sub system. It's a GenServer
-based hub that manages subscriptions, routes signals, and maintains a comprehensive signal log for replay capabilities.
Bus Architecture
The bus operates as a centralized message broker that:
- Maintains active subscriptions and their routing rules
- Logs every signal for replay and audit purposes
- Handles both synchronous and asynchronous signal delivery
- Provides middleware integration points
- Supports persistent subscriptions with acknowledgment
Managing Subscriptions
Basic Subscription
alias Jido.Signal.Bus
# Subscribe to specific signal types
Bus.subscribe(:my_bus, "user.created", dispatch: {:pid, target: self()})
# Subscribe with wildcards
Bus.subscribe(:my_bus, "user.*", dispatch: {:pid, target: self()})
Bus.subscribe(:my_bus, "payment.**", dispatch: {:pid, target: self()})
Subscription Options
Bus.subscribe(:my_bus, "user.*", [
dispatch: {:pid, target: self()},
priority: 100, # Higher priority handlers run first
persistent?: true, # Enable persistent delivery
filter: fn signal -> # Optional signal filtering
signal.data.active == true
end
])
Unsubscribing
# Get subscription ID when subscribing
{:ok, sub_id} = Bus.subscribe(:my_bus, "user.*", dispatch: {:pid, target: self()})
# Unsubscribe using the ID
Bus.unsubscribe(:my_bus, sub_id)
Publishing Signals
Basic Publishing
signal = Jido.Signal.new(%{
type: "user.created",
source: "user_service",
data: %{user_id: 123}
})
Bus.publish(:my_bus, signal)
Publishing Options
# Publish with metadata
Bus.publish(:my_bus, signal, %{priority: :high, async: true})
# Publish multiple signals
signals = [signal1, signal2, signal3]
Bus.publish_batch(:my_bus, signals)
The Internal Signal Log
The bus maintains a comprehensive log of every signal that passes through it:
Log Structure
%{
signal: %Jido.Signal{}, # The original signal
timestamp: ~U[2024-01-01 12:00:00Z], # When it was logged
metadata: %{}, # Additional context
cause_id: "parent-signal-id" # Causality tracking
}
Querying the Log
# Get recent signals
{:ok, entries} = Bus.get_log(:my_bus, limit: 10)
# Get signals by path pattern
{:ok, entries} = Bus.get_log(:my_bus, path: "user.*", limit: 50)
# Get signals since a timestamp
since = DateTime.add(DateTime.utc_now(), -3600, :second)
{:ok, entries} = Bus.get_log(:my_bus, since: since)
Replaying History
The replay system allows you to retrieve and reprocess signals from the log:
Basic Replay
# Replay all signals matching a pattern
Bus.replay(:my_bus, "user.*",
since: DateTime.add(DateTime.utc_now(), -1800, :second),
dispatch: {:pid, target: self()}
)
Advanced Replay Options
Bus.replay(:my_bus, "payment.**", [
since: ~U[2024-01-01 00:00:00Z],
until: ~U[2024-01-01 23:59:59Z],
limit: 1000,
dispatch: {:pid, target: replay_handler_pid},
filter: fn entry ->
entry.signal.data.amount > 1000
end
])
Replay to Multiple Destinations
Bus.replay(:my_bus, "user.*", [
since: DateTime.add(DateTime.utc_now(), -3600, :second),
dispatch: [
{:pid, target: processor1_pid},
{:webhook, url: "https://api.example.com/webhook"},
{:logger, level: :info}
]
])
Persistent Subscriptions
Persistent subscriptions ensure "at-least-once" delivery even if the subscriber temporarily disconnects:
Creating Persistent Subscriptions
{:ok, sub_id} = Bus.subscribe(:my_bus, "critical.alerts", [
dispatch: {:pid, target: self()},
persistent?: true,
subscriber_id: "alert_processor_1" # Stable identifier
])
Acknowledgment Flow
defmodule AlertProcessor do
use GenServer
def handle_info({:signal, signal, delivery_info}, state) do
case process_alert(signal) do
:ok ->
# Acknowledge successful processing
Bus.ack(:my_bus, delivery_info.subscription_id, delivery_info.message_id)
{:noreply, state}
{:error, reason} ->
# Don't acknowledge - signal will be redelivered
Logger.error("Failed to process alert: #{inspect(reason)}")
{:noreply, state}
end
end
defp process_alert(signal) do
# Your processing logic here
:ok
end
end
Handling Reconnections
# Reconnect and replay missed signals
Bus.reconnect(:my_bus, "alert_processor_1", [
dispatch: {:pid, target: self()},
replay_missed: true
])
Bus Configuration
Starting with Custom Options
{Jido.Signal.Bus, [
name: :my_bus,
log_retention: :timer.hours(24), # Keep logs for 24 hours
max_log_size: 100_000, # Maximum log entries
middleware: [MyApp.AuditMiddleware], # Custom middleware
serializer: Jido.Signal.Serialization.JsonSerializer
]}
Runtime Configuration
# Update bus configuration
Bus.configure(:my_bus, log_retention: :timer.hours(48))
# Get current configuration
config = Bus.get_config(:my_bus)
Monitoring and Introspection
Bus Status
status = Bus.status(:my_bus)
# Returns:
# %{
# active_subscriptions: 15,
# log_size: 1250,
# uptime: 3600,
# last_signal: ~U[2024-01-01 12:30:00Z]
# }
Subscription Listing
subscriptions = Bus.list_subscriptions(:my_bus)
# Returns list of active subscriptions with their patterns and options
Error Handling
Delivery Failures
# Configure delivery retry behavior
Bus.subscribe(:my_bus, "user.*", [
dispatch: {:http, url: "https://api.example.com/webhook"},
retry_policy: %{
max_retries: 3,
backoff: :exponential,
base_delay: 1000
}
])
Dead Letter Handling
# Signals that fail delivery can be routed to a dead letter queue
Bus.subscribe(:my_bus, "user.*", [
dispatch: {:pid, target: self()},
dead_letter: {:logger, level: :error}
])
Performance Considerations
Batch Operations
For high-throughput scenarios, use batch operations:
# Batch publish multiple signals
signals = Enum.map(1..1000, fn i ->
Jido.Signal.new(%{
type: "batch.item",
source: "batch_processor",
data: %{item_id: i}
})
end)
Bus.publish_batch(:my_bus, signals)
Asynchronous Processing
# Publish without blocking
Bus.publish_async(:my_bus, signal)
# Subscribe with async dispatch
Bus.subscribe(:my_bus, "user.*", [
dispatch: {:pid, target: self(), async: true}
])
The Signal Bus provides a robust foundation for building scalable, observable event-driven systems with comprehensive replay and persistence capabilities.