Requests And Responses

Copy Markdown View Source

Inference.Request and Inference.Response are deliberately plain structs. They are meant to be easy to inspect, trace, redact, and convert to provider payloads.

Requests

Build a request from prompt text:

{:ok, request} = Inference.Request.from_prompt("Explain the route.")

Build a request from messages:

{:ok, request} =
  Inference.Request.from_messages([
    %{role: :system, content: "Be terse."},
    %{role: :user, content: "Explain the route."}
  ])

Valid roles are:

  • :system
  • :user
  • :assistant
  • :tool

Invalid roles and empty content fail before adapter dispatch.

Responses

Adapters return Inference.Response:

{:ok, response} = Inference.complete(client, request)
text = Inference.Response.text(response)

The response carries:

  • provider and model identifiers;
  • normalized text;
  • optional object output and tool calls;
  • usage, cost, and finish reason;
  • raw provider result for in-memory use;
  • metadata and trace summary.

Do not persist raw provider responses by default. Persist redacted trace and summary fields instead.

cost is provider-reported only. Adapters should copy cost data when the underlying provider returns it, but they must not invent cost values for providers that do not expose them.