ExLLM.ResponseCache (ex_llm v0.5.0)
View SourceResponse caching system for collecting and storing real provider responses.
This module allows ExLLM to cache actual responses from providers like OpenAI, Anthropic, OpenRouter, etc., and later replay them in tests using the Mock adapter.
Usage
Automatic Response Caching
Enable caching in your environment:
# Cache all responses during testing
export EX_LLM_CACHE_RESPONSES=true
export EX_LLM_CACHE_DIR="/path/to/cache/responses"
Manual Response Caching
# Store a response
ExLLM.ResponseCache.store_response("openai", request_key, response_data)
# Retrieve a cached response
cached = ExLLM.ResponseCache.get_response("openai", request_key)
# Load all responses for a provider
responses = ExLLM.ResponseCache.load_provider_responses("anthropic")
Integration with Mock Adapter
# Configure mock to use cached responses from OpenAI
ExLLM.ResponseCache.configure_mock_provider("openai")
# Use cached responses in tests
{:ok, response} = ExLLM.chat(messages, provider: :mock)
Cache Structure
Responses are stored in JSON files organized by provider:
cache/
├── openai/
│ ├── chat_completions.json
│ ├── embeddings.json
│ └── streaming.json
├── anthropic/
│ ├── messages.json
│ └── streaming.json
└── openrouter/
├── chat_completions.json
└── models.json
Each file contains an array of cached request/response pairs with metadata.
Summary
Functions
Returns the cache directory path.
Returns true if response caching is enabled.
Clears all cached responses.
Clears all cached responses for a provider.
Configures the Mock adapter to use cached responses from a specific provider.
Retrieves a cached response matching the request.
Lists all available providers with cached responses.
Loads all cached responses for a provider.
Stores a response from the unified cache system.
Stores a response in the cache.
Functions
Returns the cache directory path.
Returns true if response caching is enabled.
Clears all cached responses.
Clears all cached responses for a provider.
Configures the Mock adapter to use cached responses from a specific provider.
Retrieves a cached response matching the request.
Lists all available providers with cached responses.
Loads all cached responses for a provider.
Stores a response from the unified cache system.
This is called by ExLLM.Cache when disk persistence is enabled.
Stores a response in the cache.