Plexus.Belief.Calibration (Plexus v0.1.0)

Copy Markdown View Source

Calibration diagnostics and a dependency-free isotonic calibrator.

Samples are {predicted_probability, observed_boolean} pairs. The reliability output is intended for experiment artifacts; the isotonic model is monotone and can be applied at runtime without treating raw noul values as calibrated likelihoods.

Summary

Types

model()
sample()

Functions

apply(map, p)
brier_score(samples)
expected_calibration_error(samples, bins \\ 10)
fit_isotonic(samples)
log_loss(samples)
reliability(samples, bins \\ 10)

Types

model()

@type model() :: %{method: :isotonic, blocks: [map()]}

sample()

@type sample() :: {number(), boolean() | 0 | 1}

Functions

apply(map, p)

@spec apply(model(), number()) :: float()

brier_score(samples)

@spec brier_score([sample()]) :: float()

expected_calibration_error(samples, bins \\ 10)

@spec expected_calibration_error([sample()], pos_integer()) :: float()

fit_isotonic(samples)

@spec fit_isotonic([sample()]) :: model()

log_loss(samples)

@spec log_loss([sample()]) :: float()

reliability(samples, bins \\ 10)

@spec reliability([sample()], pos_integer()) :: [map()]