Foresight.Evals.RecallQuality (Foresight v0.1.0)

Copy Markdown View Source

QUARANTINED (Track B). This is the legacy phrase-matching grader. It scores answers by checking for the presence of specific corpus phrases, which is exactly the signal the production ranking code was once gamed to satisfy — so its numbers measure phrase echo, not retrieval quality, and MUST NOT be cited as a quality metric.

It is retained only as a fixed regression fixture (its scenarios and text matchers are frozen and unit-tested for their own stable behaviour). Honest recall quality is measured by Foresight.Evals.RankingMetrics over the held-out corpus (Foresight.TestSupport.Evals.HoldoutCorpus), which scores id-based ranking and cannot be satisfied by emitting a phrase. Do not add scenarios here; extend the held-out corpus.

Summary

Functions

acceptable_texts(map)

corpus()

matches_reflect_text?(scenario, text)

matches_text?(scenario, text)

reflect_corpus()

reflect_scenarios()

required_memory_texts(arg1)

scenarios()