Aludel.Evals.JudgeCatalog (aludel v0.7.0)

Copy Markdown View Source

Versioned rubric templates for common model-backed evaluation tasks.

Templates provide stable IDs and criteria while allowing each assertion to choose its provider and pass threshold. The template version is recorded in metric results so historical runs retain the criteria they used.

Summary

Types

id()
template()

Functions

all()

Returns all templates in their display order.

fetch(id)

Fetches a template by its stable ID.

ids()

Returns the stable template IDs in display order.

Types

id()

@type id() :: String.t()

template()

@type template() :: %{
  id: id(),
  name: String.t(),
  description: String.t(),
  version: pos_integer(),
  rubric: String.t()
}

Functions

all()

@spec all() :: [template()]

Returns all templates in their display order.

fetch(id)

@spec fetch(term()) :: {:ok, template()} | :error

Fetches a template by its stable ID.

ids()

@spec ids() :: [id()]

Returns the stable template IDs in display order.