Imp.Clients.ReqLLM (Imp v0.5.0)

Copy Markdown View Source

Imp LM client backed by the Elixir req_llm ecosystem.

Imp owns signatures, adapters, optimizers, traces, and evaluation. req_llm owns provider/model resolution, Req/Finch transport, streaming, provider option translation, and canonical response structs.

:input_envelope is an Imp-owned safety option. It accepts a positive :max_bytes guard and an optional positive :reservation_tokens value. Imp measures the rendered message content before cache lookup or transport, raises Imp.OperationalSafetyError when the byte guard is exceeded, and removes the envelope before calling ReqLLM. The token value records a pricing or capacity reservation; without a model tokenizer it is not treated as an exact token counter.

A failed request returns {:error, %Imp.LMError{}}, whether the provider answered with an error status, the connection failed, or ReqLLM raised; the struct carries the status, whether a retry may succeed, and whether the input was longer than the model's context window, with ReqLLM's own error under :reason. Only an HTTP 400 with the structured code error.code = "context_length_exceeded" counts as the context window; prose and generic HTTP 400 responses do not trigger context recovery. A successful HTTP response whose body carries a provider error, which is how OpenRouter relays an upstream refusal, is returned as that error, never as an empty completion.

:reasoning_effort is the one reasoning option, on the client or on a call. It takes none, minimal, low, medium, high, xhigh or default, as an atom or a string. A call naming nil spends no reasoning on that call whatever the client is configured with. Native reasoning fields (Imp.Predict) set the same option, so a client configured with an effort and a program that asks for one never disagree.

OpenRouter accepts the effort in two wire fields, and its endpoint catalog says which one an endpoint supports: ReqLLM's top-level reasoning_effort (the default here) or the nested reasoning object, which openrouter_reasoning_wire: :nested selects. The switch names an encoding only; the value is always :reasoning_effort.

Summary

Types

t()

@type t() :: %Imp.Clients.ReqLLM{
  model: ReqLLM.model_input(),
  opts: keyword(),
  req_module: module()
}

Functions

cache_key(lm, messages, opts)

dump(lm)

generate_async(lm, messages, opts \\ [])

new(model_spec, opts \\ [])