ExLLM.Providers.Gemini.Caching (ex_llm v0.8.1)

View Source

Google Gemini Context Caching API implementation.

Provides functionality to cache and reuse large contexts across multiple requests, reducing costs and improving performance for repeated queries on the same content.

Summary

Functions

Creates a new cached content resource.

Deletes cached content.

Gets a specific cached content.

Lists cached contents.

Updates cached content (only expiration can be updated).

Types

create_options()

@type create_options() :: [{:config_provider, pid() | atom()}]

list_options()

@type list_options() :: [
  page_size: integer(),
  page_token: String.t(),
  config_provider: pid() | atom()
]

update_options()

@type update_options() :: [update_mask: String.t(), config_provider: pid() | atom()]

Functions

create_cached_content(request, opts \\ [])

@spec create_cached_content(map(), create_options()) ::
  {:ok, ExLLM.Providers.Gemini.Caching.CachedContent.t()} | {:error, term()}

Creates a new cached content resource.

Parameters

  • request - Map containing:
    • :contents - Content to cache
    • :model - Model to use (required)
    • :ttl or :expire_time - Expiration (one required)
    • :display_name - Optional display name
    • :system_instruction - Optional system instruction
    • :tools - Optional tools
    • :tool_config - Optional tool config
  • opts - Options including :config_provider

Examples

request = %{
  contents: [%Content{role: "user", parts: [%Part{text: "Large context"}]}],
  model: "models/gemini-2.0-flash",
  ttl: "3600s",
  display_name: "My Cache"
}
{:ok, cached} = ExLLM.Providers.Gemini.Caching.create_cached_content(request)

delete_cached_content(name, opts \\ [])

@spec delete_cached_content(String.t(), Keyword.t()) :: :ok | {:error, term()}

Deletes cached content.

Parameters

  • name - The cached content name
  • opts - Options including :config_provider

Examples

:ok = ExLLM.Providers.Gemini.Caching.delete_cached_content("cachedContents/abc-123")

get_cached_content(name, opts \\ [])

@spec get_cached_content(String.t(), Keyword.t()) ::
  {:ok, ExLLM.Providers.Gemini.Caching.CachedContent.t()} | {:error, term()}

Gets a specific cached content.

Parameters

  • name - The cached content name (e.g., "cachedContents/abc-123")
  • opts - Options including :config_provider

Examples

{:ok, cached} = ExLLM.Providers.Gemini.Caching.get_cached_content("cachedContents/abc-123")

list_cached_contents(opts \\ [])

@spec list_cached_contents(list_options()) ::
  {:ok,
   %{
     cached_contents: [ExLLM.Providers.Gemini.Caching.CachedContent.t()],
     next_page_token: String.t() | nil
   }}
  | {:error, term()}

Lists cached contents.

Parameters

  • opts - Options including :page_size, :page_token, and :config_provider

Examples

{:ok, %{cached_contents: contents}} = ExLLM.Providers.Gemini.Caching.list_cached_contents(page_size: 10)

update_cached_content(name, update, opts \\ [])

@spec update_cached_content(String.t(), map(), update_options()) ::
  {:ok, ExLLM.Providers.Gemini.Caching.CachedContent.t()} | {:error, term()}

Updates cached content (only expiration can be updated).

Parameters

  • name - The cached content name
  • update - Map containing either :ttl or :expire_time
  • opts - Options including :update_mask and :config_provider

Examples

{:ok, updated} = ExLLM.Providers.Gemini.Caching.update_cached_content(
  "cachedContents/abc-123",
  %{ttl: "7200s"}
)