SkillKit.LLM.Anthropic (SkillKit v0.4.0)

Copy Markdown View Source

Anthropic adapter for SkillKit.LLM.

Sensitive config (api_key, endpoint) is resolved from opts first (for direct test calls), then app config, then env vars. Request params (model, max_tokens, temperature) are read from opts.

Prompt caching

Prompt caching is on by default. Two cache breakpoints are placed on every request:

  1. System prompt breakpoint — added to the last block of the system prompt, caching the tool list and system content together (the longest stable prefix).
  2. Last-message breakpoint — added to the final user message, providing a rolling cache that captures the conversation history up to that turn.

Because caching is a byte-exact prefix match, the system prompt and tool list must be byte-identical across requests for cache reads to occur — keep the tool set and its ordering stable across calls.

Disabling caching

Add to config/config.exs (or an environment-specific config):

config :skill_kit, SkillKit.LLM.Anthropic, cache: false

Alternatively, pass cache: false in the opts to stream/2 directly.

TTL

The default cache TTL is "5m" (5 minutes, Anthropic's standard tier). To use the extended 1-hour TTL (requires the prompt_caching_2024_07_31 beta on your account):

config :skill_kit, SkillKit.LLM.Anthropic, cache_ttl: "1h"

Or pass cache_ttl: "1h" in opts.

Summary

Functions

build_request(messages, opts)

Builds the encoded messages and request options for an Anthropic call, applying prompt-cache breakpoints unless cache: false.

Functions

build_request(messages, opts)

@spec build_request(
  [SkillKit.LLM.message()],
  keyword()
) :: {[map()], keyword()}

Builds the encoded messages and request options for an Anthropic call, applying prompt-cache breakpoints unless cache: false.

Returns {encoded_messages, request_opts}.