LlamaCppEx.ChatCompletionChunk (LlamaCppEx v0.8.63)

Copy Markdown View Source

OpenAI-compatible streaming chat completion chunk struct.

Mirrors the shape of server-sent events from POST /v1/chat/completions with stream: true.

Summary

Types

t()

A fragment of a tool call. Its first delta carries :id, :type and the function :name; later ones only :index and more of :arguments.

Types

chunk_choice()

@type chunk_choice() :: %{
  index: integer(),
  delta: %{
    optional(:role) => String.t(),
    optional(:content) => String.t(),
    optional(:reasoning_content) => String.t(),
    optional(:tool_calls) => [tool_call_delta()]
  },
  finish_reason: String.t() | nil
}

t()

@type t() :: %LlamaCppEx.ChatCompletionChunk{
  choices: [chunk_choice()],
  created: integer(),
  id: String.t(),
  model: String.t(),
  object: String.t()
}

tool_call_delta()

@type tool_call_delta() :: %{
  :index => non_neg_integer(),
  optional(:id) => String.t(),
  optional(:type) => String.t(),
  function: %{optional(:name) => String.t(), arguments: String.t()}
}

A fragment of a tool call. Its first delta carries :id, :type and the function :name; later ones only :index and more of :arguments.