Aludel.Evals.TestCase (aludel v0.6.1)

Copy Markdown View Source

Schema for individual test cases within a suite.

Test cases define prompt variable values, optional multi-turn messages, metadata, document attachments, and assertions used to validate LLM output. Cases imported from a reusable dataset retain source provenance.

Supported assertion types:

  • contains: Check if output contains a specific string
  • not_contains: Check if output does not contain a string
  • regex: Match output against a regular expression
  • exact_match: Check for exact string match
  • json_field: Compare a typed value at a dot-separated JSON path
  • json_deep_compare: Score an expected JSON structure with an optional threshold

Summary

Types

t()

Functions

changeset(test_case, attrs)

Changeset for creating or updating a test case.

Types

t()

@type t() :: %Aludel.Evals.TestCase{
  __meta__: term(),
  assertions: term(),
  documents: term(),
  id: term(),
  inserted_at: term(),
  messages: term(),
  metadata: term(),
  source_dataset_entry: term(),
  source_dataset_entry_id: term(),
  suite: term(),
  suite_id: term(),
  updated_at: term(),
  variable_values: term()
}

Functions

changeset(test_case, attrs)

@spec changeset(t(), map()) :: Ecto.Changeset.t()

Changeset for creating or updating a test case.

Validates that suite_id, variable_values, and assertions are present.