LLMUtils.Client (llm_utils v0.1.3)

Copy Markdown View Source

Core LLM HTTP client for OpenAI-compatible chat completion APIs.

Handles:

  • Provider config resolution (by string ID or config map)
  • API key resolution from env or override
  • Request body construction with model, messages, temperature, json_mode, tools
  • Rate limiting via LLMUtils.RateLimiter (configurable)
  • Circuit breaking via LLMUtils.CircuitBreaker (configurable)
  • HTTP execution via Req.post
  • Response normalization (content extraction, tool calls, error handling)
  • Provider-specific handling (thinking tags, system→user conversion, reasoning content)
  • Metrics collection via LLMUtils.Metrics (configurable)
  • Structured logging via LLMUtils.Logging (configurable)
  • Fallback to alternative providers on failure

Override callbacks

All stateful features use ETS-based defaults but can be overridden:

LLMUtils.Client.chat_completion(messages, "openai",
  circuit_breaker: MyApp.CircuitBreaker,
  metrics_collector: MyApp.Metrics,
  rate_limiter: MyApp.RateLimiter,
  logger: MyApp.Logging
)

Summary

Functions

chat_completion(messages, provider, opts \\ [])

Send a chat completion request.

Functions

chat_completion(messages, provider, opts \\ [])

@spec chat_completion([map()], String.t() | map(), keyword()) ::
  {:ok, map()} | {:error, term()}

Send a chat completion request.

Parameters

  • messages — List of message maps with role and content keys
  • provider — Provider string ID (e.g. "openai") or a provider config map
  • opts — Keyword options

Options

  • :model — Model override (default: provider's default_model)
  • :api_key — API key override (default: from env via Provider.get_api_key/1)
  • :temperature — Temperature override (default: 0.0)
  • :json_mode — Force JSON response format (default: from provider config)
  • :tools — List of tool definitions for function calling
  • :timeout — Request timeout in ms (default: 60_000)
  • :max_tokens — Max tokens in response (default: nil = no limit)
  • :suppress_thinking — Disable thinking/reasoning (default: from provider config)
  • :fallback — List of fallback provider IDs to try on failure
  • :enable_circuit_breaker — Enable circuit breaker (default: true)
  • :enable_rate_limiter — Enable rate limiting (default: true)
  • :enable_metrics — Enable metrics collection (default: true)
  • :enable_logging — Enable structured logging (default: false)
  • :circuit_breaker — Custom circuit breaker module (default: LLMUtils.CircuitBreaker)
  • :metrics_collector — Custom metrics module (default: LLMUtils.Metrics)
  • :logger — Custom logger module (default: LLMUtils.Logging)
  • :rate_limiter — Custom rate limiter module (default: LLMUtils.RateLimiter)

Returns

  • {:ok, %{content: String.t(), usage: %{...}}}
  • {:ok, %{tool_calls: [...]}}
  • {:error, {:rate_limited, msg, retry_after}}
  • {:error, {:unauthorized, msg}}
  • {:error, {:server_error, status, msg}}
  • {:error, {:client_error, status, msg}}
  • {:error, {:precondition_failed, msg}}
  • {:error, :missing_api_key}
  • {:error, :unknown_provider}
  • {:error, :empty_response}
  • {:error, :timeout}
  • {:error, :service_unavailable}
  • {:error, :circuit_open}
  • {:error, reason} (catch-all)