Tribunal.Sampling (Tribunal v3.0.0)

Copy Markdown View Source

Reduces ordered evaluation attempts into one case result.

Sampling measures application nondeterminism. An operational error in any attempt always fails the reduced result, regardless of the quality-oriented pass rule.

Summary

Types

assertion_stat()
pass_rule()
sample()

Functions

reduce(attempts, pass_rule)

Reduces a nonempty ordered list of evaluator result maps.

Types

assertion_stat()

@type assertion_stat() :: %{
  index: non_neg_integer(),
  type: atom() | String.t(),
  passed: non_neg_integer(),
  failed: non_neg_integer(),
  errors: non_neg_integer(),
  pass_rate: float()
}

pass_rule()

@type pass_rule() :: :all | :any | :majority | {:rate, float()}

sample()

@type sample() :: %{
  repeat: pos_integer(),
  pass_rule: pass_rule(),
  passed: non_neg_integer(),
  failed: non_neg_integer(),
  errors: non_neg_integer(),
  pass_rate: float(),
  assertions: [assertion_stat()]
}

Functions

reduce(attempts, pass_rule)

@spec reduce([map()], pass_rule()) :: map()

Reduces a nonempty ordered list of evaluator result maps.

Supported pass rules are :all, :any, :majority, and {:rate, value} where value is a float between 0.0 and 1.0.

For one attempt, all existing top-level values are preserved. For repeated attempts, compatibility fields for the output and assertion results project the final attempt while status, failures, execution error, and duration are reduced across every attempt.