The Hindsight arm of the matched-reader Foresight-vs-Hindsight scoreboard.
This is the instrument that turns "we meet/exceed Hindsight" from an assertion into
a measured number. It drives a LIVE Hindsight authority over its HTTP API through
the exact same lifecycle the Foresight reflect arm runs — retain the same sessions,
trigger consolidation, then reflect — so the ONLY variable is the memory
architecture. Fairness is only real if that authority runs reflect AND consolidation
on the SAME reader model as Foresight (gpt-oss:20b); see
scripts/start-hindsight-fixture-authority.sh (it must export
HINDSIGHT_API_REFLECT_LLM_* and HINDSIGHT_API_CONSOLIDATION_LLM_*, not only
HINDSIGHT_API_LLM_*).
Routes (verified against the Hindsight OpenAPI client):
PUT /v1/default/banks/{bank}— create/reset the bankPOST /v1/default/banks/{bank}/memories— retain (async: false)POST /v1/default/banks/{bank}/consolidate— trigger consolidation (async; pollGET .../statspending_consolidationto 0)POST /v1/default/banks/{bank}/reflect— reflect →%{"text" => answer}GET /v1/default/banks/{bank}/stats—pending_consolidation, ...DELETE /v1/default/banks/{bank}— drop the bank between instances
The public core (retain_body/1, reflect_body/2, parse_reflect_answer/1,
consolidation_pending?/1, client/2) is pure and unit-tested; generate/2 is the
thin live-I/O orchestration exercised only in the matched-model :ml/:eval lane.
Summary
Functions
Build a Hindsight client from a base url (trailing slash trimmed) and api key.
Whether a bank-stats payload reports outstanding consolidation work.
Run one instance through the Hindsight arm on a fresh bank: reset → retain →
consolidate (poll to completion) → reflect. Returns {:ok, answer, meta} where
meta records consolidated? (did pending reach 0 before the timeout) and
retained (item count) so the report can be honest about whether consolidation
actually finished. Any HTTP failure short-circuits to {:error, reason}.
Whether a bank-stats payload reports outstanding retain/ingestion work.
Extract the answer text from a reflect response; missing/nil → empty string.
Reflect request body; query_timestamp is omitted entirely when nil.
Async retain body for a list of memory items. Content and every reflected field are
sent byte-for-byte identical to Foresight; the ONLY transform is that non-string
metadata VALUES are stringified, because Hindsight's retain schema rejects list/int
metadata (Foresight carries has_answer_turns as an int list). Metadata is eval
bookkeeping, not reflected content, so this keeps the head-to-head fair.
Types
Functions
Build a Hindsight client from a base url (trailing slash trimmed) and api key.
Whether a bank-stats payload reports outstanding consolidation work.
@spec generate(client(), String.t(), [map()], String.t(), String.t() | nil) :: {:ok, String.t(), map()} | {:error, term()}
Run one instance through the Hindsight arm on a fresh bank: reset → retain →
consolidate (poll to completion) → reflect. Returns {:ok, answer, meta} where
meta records consolidated? (did pending reach 0 before the timeout) and
retained (item count) so the report can be honest about whether consolidation
actually finished. Any HTTP failure short-circuits to {:error, reason}.
Whether a bank-stats payload reports outstanding retain/ingestion work.
Extract the answer text from a reflect response; missing/nil → empty string.
Reflect request body; query_timestamp is omitted entirely when nil.
Async retain body for a list of memory items. Content and every reflected field are
sent byte-for-byte identical to Foresight; the ONLY transform is that non-string
metadata VALUES are stringified, because Hindsight's retain schema rejects list/int
metadata (Foresight carries has_answer_turns as an int list). Metadata is eval
bookkeeping, not reflected content, so this keeps the head-to-head fair.
async: true because Hindsight's synchronous retain does not return reliably over
HTTP even when the ingestion completes; the async call returns instantly and the
worker does the extract/embed/link work, which generate/5 polls to completion.