API Reference Foresight v#0.1.0

Semantic memory: retain what happened, recall what is relevant, reflect to answer.

Dispatch surface for storage backends.

Execution context for the public API.

OpenAI-compatible remote embedder: POSTs to <base_url>/embeddings (LM Studio, vLLM, Ollama's OpenAI shim, or a cloud provider). Mirrors the LLM local↔cloud fork (Foresight.LLMs.LlmCore) for embeddings, so a single endpoint can serve both the reader and the embedder in dev, and a cloud provider can serve both in production — closing the "embeddings are local-only" gap.

Entity resolution scoring (C2 slice B1) — decide whether an incoming entity mention resolves to an existing bank entity or creates a new one.

Seam for deriving entity names from raw content.

LLM-backed fallback entity extractor (the last C2 slice).

The error currency of the library.

The dataset-adapter seam for the honest evaluation harness.

Counts the ways a retain pipeline silently degrades, so an eval can refuse to score a run that was not actually measuring what it claims.

The Hindsight arm of the matched-reader Foresight-vs-Hindsight scoreboard.

D2 component 2: pool the pre-fusion recall arms into the deduped candidate set to be labeled for a query.

D2: offline coordinate-ascent over per-arm fusion weights, scored as mean nDCG@k against graded relevance labels.

D2 component 3b: parse a human-filled gold artifact (see Foresight.Evals.Labeling.GoldSet.to_artifact/2) into validated, human-sourced label attrs ready for Foresight.Schemas.LabeledPair.changeset/2.

D2 component 3a: build the human-graded gold set from per-query candidate pools and serialize it to a gradeable artifact.

D2 component 4: LLM graded-relevance judge. For each (query, candidate) pair it asks a complete_json_schema/4-capable LLM (default: z.ai GLM-5.2) for a 0..3 relevance grade, and emits source: :llm label attrs recording the judge model.

D2: Cohen's kappa between human and llm calibration grades — the trust gate on the bulk judge. If the judge doesn't agree well enough with the human gold slice, its labels can't be trusted to train D2 against (optimizing to a machine's biases is the Track-B gaming trap), so the pipeline gates the full labeling run on this number.

D2: persist label attrs (from GoldIngest or the judge labeler) into labeled_pairs, idempotently. A re-run upserts on the natural key (bank_id, query_ref, memory_ref, source) rather than duplicating, so a labeling pass can be safely resumed or corrected. Invalid rows are collected, not fatal — the batch reports what it wrote and what it rejected.

Pure orchestration helpers for the D2 relevance-labeling pipeline.

Re-rank a recorded candidate pool under a trial fusion configuration, offline.

Adapter for LongMemEval (ICLR 2025), an MIT-licensed benchmark for long-term agent memory. It maps almost one-to-one onto Foresight's differentiators: temporal reasoning, knowledge updates (supersession), multi-session reasoning, and abstention.

Loader for the sanitized professional knowledge-work evaluation lane.

Pure, deterministic retrieval-quality metrics — the gaming-resistant core of the Track B eval.

QUARANTINED (Track B). This is the legacy phrase-matching grader. It scores answers by checking for the presence of specific corpus phrases, which is exactly the signal the production ranking code was once gamed to satisfy — so its numbers measure phrase echo, not retrieval quality, and MUST NOT be cited as a quality metric.

Dataset-agnostic eval runner. Given normalised Dataset.Instances and a recall_fun, it scores each query with the gaming-resistant ranking metrics and aggregates overall and per question_type.

Statistics for the answer/reflect-judge eval lane.

Fitness check: recall/reflect logic must not hardcode eval-corpus or project literals.

Fitness check: every Oban.Worker's declared queue must exist in the resolved Oban configuration.

Resolves a request into a tenancy Context, and is the authorization authority.

Seam behaviour for LLM completion.

Recover structured JSON from local-LLM output that strict schema parsing rejected.

Nx.Serving for the D4 NLI entailment verifier: a Bumblebee.Text.text_classification serving over a dedicated MNLI cross-encoder. Input is {premise, hypothesis} string pairs (evidence, clause); output is per-label softmax scores, from which EntailmentVerifiers.Bumblebee reads the entailment probability.

Faithful Elixir port of Hindsight's engine/consolidation/prompts.py (build_batch_consolidation_prompt): the LLM merge contract that turns raw facts into consolidated observations via creates/updates/deletes.

System prompt for the agentic (tool-calling) reflect loop (C6). Faithful port of Hindsight's engine/reflect/prompts.py build_system_prompt_for_tools.

Reflect agent tool definitions (C6) — the provider-neutral LlmToolkit.Tool set the agentic reflect loop exposes to the LLM. Faithful port of Hindsight's engine/reflect/tools_schema.py (get_reflect_tools).

Retain-time atomic fact extraction prompt.

Resolves the option list handed to Foresight.Recall.RRF.fuse/2.

Extracts a temporal constraint (a %{from: DateTime, to: DateTime} day/period range) from a natural-language query.

Rank/score fusion across recall arms. Three modes (D1), selected by :fusion

Sizes the rerank prefilter shortlist — how many fused candidates are handed to the (expensive) cross-encoder before the final selection.

Filter the LLM's declared citations (from the terminal done tool's args) down to those that were actually retrieved during the reflect loop .

Render a bank's disposition traits into behavioural prose for the reflect prompt (D6). Faithful port of Hindsight's engine/search/think_utils.py build_disposition_description + describe_trait_level, activating the stored disposition_skepticism / disposition_literalism / disposition_empathy fields that were otherwise inert.

Backend-agnostic grounding gate for D4. Splits a reflect answer into clauses, asks the configured EntailmentVerifier whether the cited-memory text entails each, and applies the pass/fail policy — returning the unsupported clauses so the agentic loop can re-enter targeting exactly what wasn't grounded.

Seam for the D4 grounding gate: given the clauses of a reflect answer and the text of its cited memories (evidence), return a per-clause verdict on whether the evidence entails the clause.

NLI entailment verifier (D4). Scores each (evidence, clause) pair with a dedicated MNLI cross-encoder — a calibrated, independent entailment signal, the strongest of the three verifier backends, for hosts with the ML headroom (e.g. DGX).

LLM-judge entailment verifier (D4). One structured call over the existing LLM routing decides, for each answer clause, whether the cited-memory text supports it — no new model or serving. Reuses the complete_json_schema/4 seam.

Default entailment verifier: verification OFF. Every clause is reported supported, so the D4 gate is inert until a real verifier is configured — no new cost, latency, or change to the honest baseline unless explicitly opted in.

Lexical relevance ranking + limit for the reflect search_mental_models tool .

Builds the text a cross-encoder reranker scores for a candidate.

Retain-time atomic fact extraction — the keystone that gives reflect something structured to aggregate over.

D2: graded relevance label for a (query, candidate memory) pair — the training/eval substrate for learned per-bank fusion weights.

A directed edge in the memory graph (Track C1): from_unit_id -> to_unit_id with a link_type and weight, scoped to a bank.

The stored unit of memory: a fact, item, or observation belonging to a bank.

Consolidation orchestration. Phase A (pool existing observations, run the LLM merge, embed results) runs OUTSIDE the DB transaction; Phase B applies the prepared writes, stamps sources, and refreshes mental models INSIDE one transaction. This keeps slow LLM/embedding IO off the tenant DB connection (the reflect txn-across-LLM pool-timeout class of bug).

Consolidation persistence: an Elixir port of Hindsight's engine/consolidation/consolidator.py. Turns raw world/experience facts into consolidated observation memory units via an LLM creates/updates/ deletes merge contract (Foresight.Prompts.Consolidation.V1).

Retain-time construction of the memory_links graph (Track C1).

The embeddable supervision entry point.

Faithful port of Python difflib.SequenceMatcher(None, a, b).ratio() — the Ratcliff-Obershelp similarity Hindsight uses for entity name matching (engine/entity_resolver.py:664).

Fetch an evaluation dataset split on demand. Nothing is redistributed in the repo — the data lands in priv/evals/data/, which is gitignored.

Records the FLOOR fixture set from a live hindsight v0.7.1 authority.