Imp.Confidence.Scoring behaviour (Imp v0.5.0)

Copy Markdown View Source

Source-faithful scoring strategies for correctness and joint token logprob.

These formulas follow GEPA commit 65df4325e3fb4781cf2ab17dd144d6ce2f7b98fe. A joint logprob is converted to raw confidence with exp(logprob); this value is a model score, not a calibrated probability.

Summary

Callbacks

describe(struct)
score(struct, boolean, arg3)

Functions

score(strategy, correct?, joint_logprob)

Scores correctness with a configured strategy.

Callbacks

describe(struct)

@callback describe(struct()) :: String.t()

score(struct, boolean, arg3)

@callback score(struct(), boolean(), number() | nil) :: float()

Functions

score(strategy, correct?, joint_logprob)

Scores correctness with a configured strategy.