LLMProxy.Cache behaviour (llm_proxy v0.1.0)

Behaviour for deterministic response cache adapters.

Configure one adapter with:

config :llm_proxy, cache: MyApp.LLMCache

The adapter receives a deterministic cache key and context map. It stores and returns %LLMProxy.Response{} values; external adapters can serialize the response if needed.

Summary

Types

context()

Callbacks

get(t, context)

put(t, t, context)

Types

context()

@type context() :: %{
  optional(:actor) => LLMProxy.Actor.t(),
  optional(:api_key) => map(),
  optional(:route) => atom(),
  optional(:model) => String.t(),
  optional(:provider) => module(),
  optional(:provider_name) => String.t() | nil,
  optional(:trace_id) => String.t(),
  optional(:cache_key) => String.t(),
  optional(:metadata) => map()
}

Callbacks

get(t, context)

@callback get(String.t(), context()) ::
  {:hit, LLMProxy.Response.t()} | :miss | {:error, term()}

put(t, t, context)

(optional)

@callback put(String.t(), LLMProxy.Response.t(), context()) :: :ok | {:error, term()}