Behaviour for LLM inference providers.
Each provider implements infer/2 which takes a prompt string and returns
a LangExtract.Provider.Response — the raw response text plus token usage
when the API reported it. Parsing and normalization are the caller's
responsibility.
Shared helpers for API key resolution and HTTP error mapping are provided for use by provider implementations.
Summary
Functions
Extracts common provider options (model, max_tokens, temperature, base_url) from opts with provider-specific defaults. For use by provider implementations.
Resolves an API key from opts or an environment variable.
Maps a Req response to a provider result tuple.
Merges caller-supplied :req_options into the provider's Req options.
Posts the request and parses the response inside a
[:lang_extract, :request] telemetry span (:start, :stop, and
:exception events).
Returns a pre-built HTTP client from opts, or builds one via the given function.
Types
@type error() :: :missing_api_key | :empty_response | :unauthorized | :server_error | {:bad_request, term()} | {:rate_limited, non_neg_integer() | nil} | {:api_error, pos_integer(), term()} | {:request_error, Exception.t()}
Every error a provider can return from infer/2.
The runner's retry policy dispatches on these shapes: :rate_limited
pauses globally, :server_error and :request_error consume retry
budget, everything else fails the chunk immediately.
Callbacks
@callback build_http_client(opts :: keyword()) :: {:ok, Req.Request.t()} | {:error, term()}
@callback infer(prompt :: String.t(), opts :: keyword()) :: {:ok, LangExtract.Provider.Response.t()} | {:error, error()}
Functions
@spec common_opts(keyword(), keyword()) :: %{ model: String.t(), max_tokens: pos_integer(), temperature: number() | nil, base_url: String.t() }
Extracts common provider options (model, max_tokens, temperature, base_url) from opts with provider-specific defaults. For use by provider implementations.
Resolves an API key from opts or an environment variable.
Returns {:ok, key} or {:error, :missing_api_key} if nil or empty.
For use by provider implementations.
@spec map_response( {:ok, Req.Response.t()} | {:error, Exception.t()}, (term() -> {:ok, String.t()} | {:error, :empty_response}) ) :: {:ok, String.t()} | {:error, error()}
Maps a Req response to a provider result tuple.
Delegates to extract_text for HTTP 200; maps error status codes and
network failures to standard error tuples. For use by provider implementations.
Merges caller-supplied :req_options into the provider's Req options.
Applies shared HTTP defaults first (120s receive timeout, transient
retries), then the provider's own options, then anything in :req_options
— so callers can override any Req configuration (timeouts, retry policy,
pool settings, plug for testing, etc.) without the provider needing to
know about them.
@spec request(Req.Request.t(), keyword(), map(), (term() -> {:ok, String.t()} | {:error, error()})) :: {:ok, LangExtract.Provider.Response.t()} | {:error, error()}
Posts the request and parses the response inside a
[:lang_extract, :request] telemetry span (:start, :stop, and
:exception events).
The span's duration wraps the full Req.post/2 call — including Req's
transient retries — because that is the latency the pipeline experiences.
:stop measurements additionally carry input_tokens/output_tokens
when the response body has a usage block (Anthropic/OpenAI "usage",
Gemini "usageMetadata"); handlers should treat missing keys as unknown,
not zero.
Metadata: the caller's provider and model; :stop adds status —
the HTTP status code, or :transport_error when no response arrived.
@spec resolve_http_client( keyword(), (keyword() -> {:ok, Req.Request.t()} | {:error, term()}) ) :: {:ok, Req.Request.t()} | {:error, term()}
Returns a pre-built HTTP client from opts, or builds one via the given function.