ExLLM.Infrastructure.CircuitBreaker.Bulkhead (ex_llm v0.8.1)

View Source

Bulkhead pattern implementation for circuit breaker concurrency limiting.

The bulkhead pattern isolates resources by limiting the number of concurrent requests to prevent cascading failures and resource exhaustion.

Features

  • Concurrency limiting: Enforce maximum concurrent requests per circuit
  • Request queuing: Queue requests when bulkhead is full
  • Timeout handling: Timeout queued requests after configurable period
  • Metrics tracking: Monitor active, queued, and rejected requests
  • Provider-specific limits: Different limits per provider/circuit
  • Integration: Works seamlessly with circuit breaker states

Configuration

  • max_concurrent: Maximum concurrent requests (default: 10)
  • max_queued: Maximum queued requests (default: 50)
  • queue_timeout: Queue timeout in milliseconds (default: 5000)

Usage

# Configure bulkhead for a circuit
ExLLM.Infrastructure.CircuitBreaker.Bulkhead.configure("api_circuit",
  max_concurrent: 5,
  max_queued: 20,
  queue_timeout: 3000
)

# Execute with bulkhead protection
ExLLM.Infrastructure.CircuitBreaker.call_with_bulkhead("api_circuit", fn ->
  # Your API call here
  HTTPClient.get("/api/data")
end)

Summary

Functions

Configure bulkhead settings for a circuit.

Execute a function with bulkhead protection.

Get bulkhead configuration for a circuit.

Get bulkhead metrics for a circuit.

Initialize the bulkhead system.

Functions

configure(circuit_name, opts)

Configure bulkhead settings for a circuit.

execute(circuit_name, fun, opts \\ [])

Execute a function with bulkhead protection.

This is the main entry point that combines circuit breaker and bulkhead logic.

get_config(circuit_name)

Get bulkhead configuration for a circuit.

get_metrics(circuit_name)

Get bulkhead metrics for a circuit.

init()

Initialize the bulkhead system.