LangEx.LLM.Anthropic (LangEx v0.7.0)

Copy Markdown View Source

Streaming Anthropic Claude adapter with tool use, thinking, and prompt caching.

Uses stream: true by default so data flows continuously, resetting the TCP receive timeout with each SSE chunk and avoiding idle-connection timeouts from intermediary infrastructure.

Features

  • Streaming — SSE-based streaming avoids TCP idle timeouts on long requests
  • Adaptive thinking — pass thinking: true to enable Claude's extended thinking; stream thinking deltas in real time via the on_thinking callback
  • Prompt caching — system prompts and the last tool definition are annotated with cache_control for Anthropic's prompt caching beta
  • Usage tracking — chat_with_usage/2 returns token counts including cache metrics
  • Model-aware defaults — max_tokens defaults based on model family

Tool Calling

Pass :tools (list of %LangEx.Tool{}) to enable tool calling. The adapter returns {:ok, %Message.AI{tool_calls: [...]}} when the model requests tool use. Use LangEx.Tool.Node to execute them.

LangEx.LLM.Anthropic.chat(messages,
  model: "claude-sonnet-4-20250514",
  tools: [%LangEx.Tool{name: "get_weather", ...}]
)

Options

  • :thinking — enable adaptive thinking (true / false, default false)
  • :on_thinking — fn(accumulated_thinking_text) -> any() callback
  • :on_token — fn(text_delta) -> any() callback invoked per streamed content token (used by graph streaming's :messages mode)
  • :prompt_caching — enable prompt caching headers (default true)
  • :stream — use SSE streaming (default true); set false for simple requests
  • :max_tokens — override max tokens (defaults: 64K for sonnet, 128K otherwise)