LangEx.LLM.Anthropic (LangEx v0.11.3)

Copy Markdown View Source

Streaming Anthropic Claude adapter with tool use, thinking, and prompt caching.

Uses stream: true by default so data flows continuously, resetting the TCP receive timeout with each SSE chunk and avoiding idle-connection timeouts from intermediary infrastructure.

Features

  • Streaming — SSE-based streaming avoids TCP idle timeouts on long requests
  • Adaptive thinking — pass thinking: true to enable Claude's extended thinking; stream thinking deltas in real time via the on_thinking callback
  • Prompt caching — system prompts, the last tool definition, and a rolling breakpoint on the last conversation message are annotated with cache_control, so a long agent loop reuses its cached message prefix each turn
  • Usage tracking — chat_with_usage/2 returns token counts including cache metrics
  • Model-aware defaults — max_tokens defaults based on model family

Tool Calling

Pass :tools (list of %LangEx.Tool{}) to enable tool calling. The adapter returns {:ok, %Message.AI{tool_calls: [...]}} when the model requests tool use. Use LangEx.Tool.Node to execute them.

LangEx.LLM.Anthropic.chat(messages,
  model: "claude-sonnet-4-20250514",
  tools: [%LangEx.Tool{name: "get_weather", ...}]
)

Options

  • :thinking — enable adaptive thinking (true / false, default false)
  • :on_thinking — fn(accumulated_thinking_text) -> any() callback
  • :on_token — fn(text_delta) -> any() callback invoked per streamed content token (used by graph streaming's :messages mode)
  • :prompt_caching — enable prompt caching headers (default true)
  • :cache_conversation — also mark a rolling cache_control breakpoint on the last conversation message (default true; only applies when :prompt_caching is on)
  • :tool_choice — force tool use: :auto (default), :required/:any (must call some tool), or {:tool, name} (must call that tool)
  • :stream — use SSE streaming (default true); set false for simple requests
  • :max_tokens — override max tokens (defaults: 64K for sonnet, 128K otherwise)