LlamaCppEx.ChatCompletion (LlamaCppEx v0.8.71)

Copy Markdown View Source

OpenAI-compatible chat completion response struct.

Mirrors the shape of POST /v1/chat/completions responses.

Summary

Types

t()

A tool call, present on message only when the reply called a tool (see the :tools option of LlamaCppEx.chat_completion/3). arguments is JSON text, as in the OpenAI API.

Types

choice()

@type choice() :: %{
  index: integer(),
  message: %{
    :role => String.t(),
    :content => String.t() | nil,
    :reasoning_content => String.t() | nil,
    optional(:tool_calls) => [tool_call()]
  },
  finish_reason: String.t()
}

t()

@type t() :: %LlamaCppEx.ChatCompletion{
  choices: [choice()],
  created: integer(),
  id: String.t(),
  model: String.t(),
  object: String.t(),
  usage: usage()
}

tool_call()

@type tool_call() :: %{
  id: String.t(),
  type: String.t(),
  function: %{name: String.t(), arguments: String.t()}
}

A tool call, present on message only when the reply called a tool (see the :tools option of LlamaCppEx.chat_completion/3). arguments is JSON text, as in the OpenAI API.

usage()

@type usage() :: %{
  prompt_tokens: integer(),
  completion_tokens: integer(),
  total_tokens: integer()
}