ExVrp.ABBenchmark.Comparator (ExVrp v0.6.0)

Copy Markdown View Source

Compares two benchmark results maps (baseline ref vs candidate ref) and decides whether the candidate regressed solution quality.

Summary

Functions

analyze(baseline, candidate)
compare(baseline, candidate)

Functions

analyze(baseline, candidate)

@spec analyze(map(), map()) :: %{
  per_instance: [ExVrp.ABBenchmark.Comparator.Row.t()],
  missing: [String.t()]
}

compare(baseline, candidate)

@spec compare(map(), map()) ::
  {:ok | :regression, %{hard_fails: [String.t()], warnings: [String.t()]}}