Aludel.Evals.TestCase (aludel v0.5.0)

Copy Markdown View Source

Schema for individual test cases within a suite.

Test cases define variable values to substitute into prompts and assertions to validate the LLM output.

Supported assertion types:

  • contains: Check if output contains a specific string
  • not_contains: Check if output does not contain a string
  • regex: Match output against a regular expression
  • exact_match: Check for exact string match

Summary

Types

t()

Functions

changeset(test_case, attrs)

Changeset for creating or updating a test case.

Types

t()

@type t() :: %Aludel.Evals.TestCase{
  __meta__: term(),
  assertions: term(),
  documents: term(),
  id: term(),
  inserted_at: term(),
  messages: term(),
  metadata: term(),
  source_dataset_entry: term(),
  source_dataset_entry_id: term(),
  suite: term(),
  suite_id: term(),
  updated_at: term(),
  variable_values: term()
}

Functions

changeset(test_case, attrs)

@spec changeset(t(), map()) :: Ecto.Changeset.t()

Changeset for creating or updating a test case.

Validates that suite_id, variable_values, and assertions are present.