LLM.Dialect behaviour (LLM v0.1.0)

Copy Markdown View Source

Behaviour for wire format translation between normalized messages and provider APIs.

Each dialect handles encoding requests to provider-specific JSON format and decoding responses/chunks back to normalized LLM structs.

Built-in Dialects

Summary

Types

chunk_type()

Callbacks

build_request(t, keyword)

Build the request body for the provider API.

decode_chunk(t)

Parse a single SSE data line into one or more chunk structs. Return :done when the stream is complete.

decode_response(map)

Decode a complete (non-streaming) response into a normalized LLM.Response.

normalize_thinking(term)

Normalize the thinking option to provider-specific format.

stream_headers(keyword)

Return extra headers needed for streaming requests.

stream_path()

Return the stream endpoint path (appended to base_url).

Types

chunk_type()

@type chunk_type() ::
  LLM.Stream.Chunk.t()
  | LLM.Stream.ToolCall.t()
  | LLM.Stream.Thinking.t()
  | LLM.Stream.Stop.t()
  | LLM.Stream.Error.t()

Callbacks

build_request(t, keyword)

@callback build_request(
  LLM.Context.t(),
  keyword()
) :: map()

Build the request body for the provider API.

decode_chunk(t)

@callback decode_chunk(String.t()) :: [chunk_type()] | :done

Parse a single SSE data line into one or more chunk structs. Return :done when the stream is complete.

decode_response(map)

@callback decode_response(map()) :: {:ok, LLM.Response.t()} | {:error, term()}

Decode a complete (non-streaming) response into a normalized LLM.Response.

normalize_thinking(term)

(optional) 
@callback normalize_thinking(term()) :: term()

Normalize the thinking option to provider-specific format.

stream_headers(keyword)

@callback stream_headers(keyword()) :: [{String.t(), String.t()}]

Return extra headers needed for streaming requests.

stream_path()

@callback stream_path() :: String.t()

Return the stream endpoint path (appended to base_url).