LangChain.ChatModels.ChatModel behaviour (LangChain v0.9.1)

Copy Markdown View Source

Summary

Callbacks

Returns the provider name for this chat model (e.g. "openai", "anthropic").

Functions

Wraps an LLM call/3 body in the standard [:langchain, :llm, :call] telemetry span with token-usage enrichment and request-parameter capture already wired.

Best-effort gen_ai.output.type for a chat model.

Returns the provider name for a given chat model struct.

Best-effort extraction of standard request parameters from a chat model struct for telemetry, using the conventional public schema field names.

Restore a ChatModel from a serialized config map.

Create a serializable map from a ChatModel's current configuration that can later be restored.

Extracts token usage from an LLM call result for use as a span/4 :enrich_stop callback.

Types

call_response()

@type call_response() ::
  {:ok,
   LangChain.Message.t()
   | [LangChain.Message.t()]
   | [LangChain.MessageDelta.t()]}
  | {:error, LangChain.LangChainError.t()}

t()

@type t() :: Ecto.Schema.t()

tool()

@type tool() :: LangChain.Function.t()

tools()

@type tools() :: [tool()]

Callbacks

call(t, arg2, list)

@callback call(
  t(),
  String.t() | [LangChain.Message.t()],
  [LangChain.Function.t()]
) :: call_response()

provider()

(optional)
@callback provider() :: String.t()

Returns the provider name for this chat model (e.g. "openai", "anthropic").

Used in telemetry metadata to identify the LLM provider without inspecting the module name. This is an optional callback — if not implemented, the provider can be derived from the module name via provider/1.

restore_from_map(map)

@callback restore_from_map(%{required(String.t()) => any()}) ::
  {:ok, struct()} | {:error, String.t()}

retry_on_fallback?(t)

@callback retry_on_fallback?(LangChain.LangChainError.t()) :: boolean()

serialize_config(t)

@callback serialize_config(t()) :: %{required(String.t()) => any()}

Functions

llm_telemetry_span(model, metadata, fun)

@spec llm_telemetry_span(struct() | nil, map(), (-> result)) :: result
when result: any()

Wraps an LLM call/3 body in the standard [:langchain, :llm, :call] telemetry span with token-usage enrichment and request-parameter capture already wired.

Every chat model must open its LLM-call span through this helper. Building the LangChain.Telemetry.span/4 call by hand in each model made it easy to forget the :enrich_stop callback — and a model that forgets it silently drops token usage from the [:langchain, :llm, :call, :stop] event (and the OTEL span) for that provider, with nothing failing to signal the mistake. Centralizing the event name and :enrich_stop here removes that footgun: a new provider only has to build its metadata and call this function.

model is the chat model struct (or nil). It is read for three pieces of telemetry metadata (each via put_new, so a provider that builds its own richer value keeps precedence):

  • :request_options — standard gen_ai.request.* parameters, via request_options/1.
  • :output_type"text" or "json", via output_type/1.
  • :endpoint — the request URL (when the model exposes :endpoint), which the OTEL layer turns into server.address / server.port.

This lets the OTEL layer emit those attributes without every provider re-plumbing them by hand. metadata is the LLM-call metadata map (:model, :provider, :message_count, :tools_count, ...). fun is the zero-arity function that performs the request and returns the call_response().

output_type(model)

@spec output_type(struct() | nil) :: String.t()

Best-effort gen_ai.output.type for a chat model.

Returns "json" when the model is configured to request structured/JSON output (:json_response set to true, a non-nil :json_schema, or a JSON-typed :response_format), otherwise "text". All LangChain chat models are chat-completion style, so only "text" and "json" are distinguished. Mirrors the conventional-field heuristic used by request_options/1.

provider(struct)

@spec provider(t()) :: String.t()

Returns the provider name for a given chat model struct.

Dispatches to the model module's provider/0 callback when implemented. Otherwise it falls back to a best-effort guess derived from the module name (drop a leading Chat, then Macro.underscore/1), e.g. LangChain.ChatModels.ChatAnthropic -> "anthropic".

The fallback is only a heuristic and cannot recover canonical names for multi-word or acronym module names — ChatOpenAIResponses derives to "open_ai_responses", not "openai_responses", so it won't round-trip through LangChain.OpenTelemetry.ProviderMapping. Every model should implement provider/0 to get a stable, canonical provider string; all in-tree models do. (A multi-provider adapter like ChatReqLLM, whose provider varies per call, instead sets :provider directly in its telemetry metadata rather than via provider/0.)

request_options(model)

@spec request_options(struct() | nil) :: map()

Best-effort extraction of standard request parameters from a chat model struct for telemetry, using the conventional public schema field names.

Reads the well-known fields (:temperature, :max_tokens, :top_p, :top_k, :frequency_penalty, :presence_penalty, :seed, :n, :stream, :stop/:stop_sequences, :reasoning_effort) when the struct defines them, dropping any that are absent or nil. This mirrors the heuristic used by provider/1: models expose these parameters as public schema fields under conventional names, so one generic reader avoids per-provider duplication. A model that names a parameter differently simply won't have it captured — the result is a useful subset, not a guarantee.

The returned map uses provider-neutral keys; LangChain.OpenTelemetry.Attributes maps them to their gen_ai.request.* semantic-convention attribute names.

restore_from_map(data)

@spec restore_from_map(nil | %{required(String.t()) => any()}) ::
  {:ok, struct()} | {:error, String.t()}

Restore a ChatModel from a serialized config map.

serialize_config(model)

Create a serializable map from a ChatModel's current configuration that can later be restored.

token_usage_from_result(arg1)

@spec token_usage_from_result(call_response()) :: %{
  token_usage: LangChain.TokenUsage.t() | nil
}

Extracts token usage from an LLM call result for use as a span/4 :enrich_stop callback.

Returns a map with :token_usage set to the %TokenUsage{} struct when available, or nil otherwise.