ReqLLM.Providers.AmazonBedrock.PromptCache (ReqLLM v1.26.0)

View Source

Prompt caching options shared by the Amazon Bedrock formatters.

Resolves the prompt_cache, prompt_cache_ttl and cache_messages provider options, builds Converse cachePoint blocks and bridges the options to the keys consumed by ReqLLM.Providers.Anthropic.

See https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html

Summary

Types

config()

Functions

apply_converse(request, config, model_id)

Place automatic checkpoints after the tools, the system prompt and the cache_messages position of an encoded Converse request, then check that no 1h checkpoint follows a 5m one.

checkpoint(ttl)

Build a Converse cachePoint block. ttl is omitted when nil.

checkpoint?(arg1)

True for a Converse cachePoint block.

explicit_checkpoint(arg1)

Return the cachePoint block declared by the cache_control metadata of a ContentPart or a Message, or nil. Only the ttl is carried over.

resolve(opts)

Resolve the prompt cache options into a config map.

to_anthropic_opts(opts)

Rewrite the generic options into the anthropic_* keys read by the Anthropic provider's prompt caching helper.

Types

config()

@type config() :: %{
  enabled: boolean(),
  ttl: String.t() | nil,
  message_offset: integer() | nil
}

Functions

apply_converse(request, config, model_id)

@spec apply_converse(map(), config(), String.t() | nil) :: map()

Place automatic checkpoints after the tools, the system prompt and the cache_messages position of an encoded Converse request, then check that no 1h checkpoint follows a 5m one.

checkpoint(ttl)

@spec checkpoint(String.t() | nil) :: map()

Build a Converse cachePoint block. ttl is omitted when nil.

checkpoint?(arg1)

@spec checkpoint?(any()) :: boolean()

True for a Converse cachePoint block.

explicit_checkpoint(arg1)

@spec explicit_checkpoint(any()) :: map() | nil

Return the cachePoint block declared by the cache_control metadata of a ContentPart or a Message, or nil. Only the ttl is carried over.

resolve(opts)

@spec resolve(keyword()) :: config()

Resolve the prompt cache options into a config map.

Top-level options win over provider_options, and the generic keys win over their anthropic_* aliases.

to_anthropic_opts(opts)

@spec to_anthropic_opts(keyword()) :: keyword()

Rewrite the generic options into the anthropic_* keys read by the Anthropic provider's prompt caching helper.