AI response cache for reducing costs and improving performance.

Caches AI provider responses using ETS with TTL-based expiration and LRU eviction.

Configuration

config :ragex, :ai_cache,
  enabled: true,
  ttl: 3600,  # 1 hour in seconds
  max_size: 1000,  # maximum cache entries
  operation_caches: %{
    query: %{ttl: 3600, max_size: 500},
    explain: %{ttl: 7200, max_size: 300},
    suggest: %{ttl: 1800, max_size: 200}
  }

Cache Key Generation

Cache keys are SHA256 hashes of:

• Query/prompt text
• Context (if provided)
• Model name
• Provider
• Relevant parameters (temperature, etc.)

Features

• TTL-based expiration
• LRU eviction when max size exceeded
• Separate caches per operation type
• Cache hit/miss metrics
• Thread-safe ETS operations