ExLLM.Providers.Gemini.Caching (ex_llm v0.8.1)
View SourceGoogle Gemini Context Caching API implementation.
Provides functionality to cache and reuse large contexts across multiple requests, reducing costs and improving performance for repeated queries on the same content.
Summary
Functions
Creates a new cached content resource.
Deletes cached content.
Gets a specific cached content.
Lists cached contents.
Updates cached content (only expiration can be updated).
Types
Functions
@spec create_cached_content(map(), create_options()) :: {:ok, ExLLM.Providers.Gemini.Caching.CachedContent.t()} | {:error, term()}
Creates a new cached content resource.
Parameters
request
- Map containing::contents
- Content to cache:model
- Model to use (required):ttl
or:expire_time
- Expiration (one required):display_name
- Optional display name:system_instruction
- Optional system instruction:tools
- Optional tools:tool_config
- Optional tool config
opts
- Options including:config_provider
Examples
request = %{
contents: [%Content{role: "user", parts: [%Part{text: "Large context"}]}],
model: "models/gemini-2.0-flash",
ttl: "3600s",
display_name: "My Cache"
}
{:ok, cached} = ExLLM.Providers.Gemini.Caching.create_cached_content(request)
Deletes cached content.
Parameters
name
- The cached content nameopts
- Options including:config_provider
Examples
:ok = ExLLM.Providers.Gemini.Caching.delete_cached_content("cachedContents/abc-123")
@spec get_cached_content(String.t(), Keyword.t()) :: {:ok, ExLLM.Providers.Gemini.Caching.CachedContent.t()} | {:error, term()}
Gets a specific cached content.
Parameters
name
- The cached content name (e.g., "cachedContents/abc-123")opts
- Options including:config_provider
Examples
{:ok, cached} = ExLLM.Providers.Gemini.Caching.get_cached_content("cachedContents/abc-123")
@spec list_cached_contents(list_options()) :: {:ok, %{ cached_contents: [ExLLM.Providers.Gemini.Caching.CachedContent.t()], next_page_token: String.t() | nil }} | {:error, term()}
Lists cached contents.
Parameters
opts
- Options including:page_size
,:page_token
, and:config_provider
Examples
{:ok, %{cached_contents: contents}} = ExLLM.Providers.Gemini.Caching.list_cached_contents(page_size: 10)
@spec update_cached_content(String.t(), map(), update_options()) :: {:ok, ExLLM.Providers.Gemini.Caching.CachedContent.t()} | {:error, term()}
Updates cached content (only expiration can be updated).
Parameters
name
- The cached content nameupdate
- Map containing either:ttl
or:expire_time
opts
- Options including:update_mask
and:config_provider
Examples
{:ok, updated} = ExLLM.Providers.Gemini.Caching.update_cached_content(
"cachedContents/abc-123",
%{ttl: "7200s"}
)