SkillKit.Eval.Transcript (SkillKit v0.5.0)

Copy Markdown View Source

The observable result of running an eval's prompt through an agent.

Captures the final assistant response, the names of tools the agent called (in call order), and the terminal :status of the run:

  • :ok — the agent produced an assistant message
  • :error — the agent emitted an error event (:reason is set)
  • :timeout — no terminal event arrived before the deadline
  • :pending — initial state, before any events were collected

Summary

Types

status()
t()
usage()

Types

status()

@type status() :: :pending | :ok | :error | :timeout

t()

@type t() :: %SkillKit.Eval.Transcript{
  error: term(),
  response: String.t() | nil,
  status: status(),
  tool_calls: [String.t()],
  usage: usage()
}

usage()

@type usage() :: %{
  input_tokens: non_neg_integer(),
  output_tokens: non_neg_integer(),
  cache_creation_input_tokens: non_neg_integer(),
  cache_read_input_tokens: non_neg_integer()
}