API Reference Foresight v#0.3.1

Semantic memory: retain what happened, recall what is relevant, reflect to answer.

Dispatch surface for storage backends.

Execution context for the public API.

OpenAI-compatible remote embedder: POSTs to <base_url>/embeddings (LM Studio, vLLM, Ollama's OpenAI shim, or a cloud provider). Mirrors the LLM local↔cloud fork (Foresight.LLMs.LlmCore) for embeddings, so a single endpoint can serve both the reader and the embedder in dev, and a cloud provider can serve both in production — closing the "embeddings are local-only" gap.

Entity resolution scoring — decide whether an incoming entity mention resolves to an existing bank entity or creates a new one.

Seam for deriving entity names from raw content.

LLM-backed fallback entity extractor.

The error currency of the library.

The dataset-adapter seam for the honest evaluation harness.

Counts the ways a retain pipeline silently degrades, so an eval can refuse to score a run that was not actually measuring what it claims.

The Hindsight arm of the matched-reader Foresight-vs-Hindsight scoreboard.

Relevance-labeling pipeline, candidate-pool step: pool the pre-fusion recall arms into the deduped candidate set to be labeled for a query.

Offline coordinate ascent over per-arm fusion weights, scored as mean nDCG@k against graded relevance labels.

Relevance-labeling pipeline, gold-ingest step: parse a human-filled gold artifact (see Foresight.Evals.Labeling.GoldSet.to_artifact/2) into validated, human-sourced label attrs ready for Foresight.Schemas.LabeledPair.changeset/2.

Relevance-labeling pipeline, gold-set step: build the human-graded gold set from per-query candidate pools and serialize it to a gradeable artifact.

Relevance-labeling pipeline, judge step: LLM graded-relevance judge. For each (query, candidate) pair it asks a complete_json_schema/4-capable LLM (default: z.ai GLM-5.2) for a 0..3 relevance grade, and emits source: :llm label attrs recording the judge model.

Cohen's kappa between human and llm calibration grades — the trust gate on the bulk judge. If the judge doesn't agree well enough with the human gold slice, its labels can't be trusted to train fusion weights against (optimizing to a machine's biases is the same metric-gaming trap the ranking metrics were built to avoid), so the pipeline gates the full labeling run on this number.

Persist label attrs (from GoldIngest or the judge labeler) into labeled_pairs, idempotently. A re-run upserts on the natural key (bank_id, query_ref, memory_ref, source) rather than duplicating, so a labeling pass can be safely resumed or corrected. Invalid rows are collected, not fatal — the batch reports what it wrote and what it rejected.

Pure orchestration helpers for the relevance-labeling pipeline.

Re-rank a recorded candidate pool under a trial fusion configuration, offline.

Adapter for LongMemEval (ICLR 2025), an MIT-licensed benchmark for long-term agent memory. It maps almost one-to-one onto Foresight's differentiators: temporal reasoning, knowledge updates (supersession), multi-session reasoning, and abstention.

Loader for the sanitized professional knowledge-work evaluation lane.

Pure, deterministic retrieval-quality metrics — the gaming-resistant core of the retrieval eval.

QUARANTINED. This is the legacy phrase-matching grader. It scores answers by checking for the presence of specific corpus phrases, which is exactly the signal the production ranking code was once gamed to satisfy — so its numbers measure phrase echo, not retrieval quality, and MUST NOT be cited as a quality metric.

Dataset-agnostic eval runner. Given normalised Dataset.Instances and a recall_fun, it scores each query with the gaming-resistant ranking metrics and aggregates overall and per question_type.

Statistics for the answer/reflect-judge eval lane.

Fitness check: recall/reflect logic must not hardcode eval-corpus or project literals.

Fitness check: every Oban.Worker's declared queue must exist in the resolved Oban configuration.

Resolves a request into a tenancy Context, and is the authorization authority.

Seam behaviour for LLM completion.

Recover structured JSON from local-LLM output that strict schema parsing rejected.

Nx.Serving for the NLI entailment verifier: a Bumblebee.Text.text_classification serving over a dedicated MNLI cross-encoder. Input is {premise, hypothesis} string pairs (evidence, clause); output is per-label softmax scores, from which EntailmentVerifiers.Bumblebee reads the entailment probability.

Installs Foresight's shared control-plane schema into a consumer database.

Classifies every migration shipped with Foresight.

Faithful Elixir port of Hindsight's engine/consolidation/prompts.py (build_batch_consolidation_prompt): the LLM merge contract that turns raw facts into consolidated observations via creates/updates/deletes.

System prompt for the agentic (tool-calling) reflect loop. Faithful port of Hindsight's engine/reflect/prompts.py build_system_prompt_for_tools.

Reflect agent tool definitions — the provider-neutral LlmToolkit.Tool set the agentic reflect loop exposes to the LLM. Faithful port of Hindsight's engine/reflect/tools_schema.py (get_reflect_tools).

Retain-time atomic fact extraction prompt.

Resolves the option list handed to Foresight.Recall.RRF.fuse/2.

Extracts a temporal constraint (a %{from: DateTime, to: DateTime} day/period range) from a natural-language query.

Rank/score fusion across recall arms. Three modes, selected by :fusion

Sizes the rerank prefilter shortlist — how many fused candidates are handed to the (expensive) cross-encoder before the final selection.

Resolves the cosine-similarity floor the semantic arm applies before it hands candidates to fusion. A unit scoring below the floor against the query embedding is never a semantic candidate, whatever the recall budget.

Filter the LLM's declared citations (from the terminal done tool's args) down to those that were actually retrieved during the reflect loop.

Render a bank's disposition traits into behavioural prose for the reflect prompt. Faithful port of Hindsight's engine/search/think_utils.py build_disposition_description + describe_trait_level, activating the stored disposition_skepticism / disposition_literalism / disposition_empathy fields that were otherwise inert.

Backend-agnostic grounding (entailment) gate for reflect answers. Splits a reflect answer into clauses, asks the configured EntailmentVerifier whether the cited-memory text entails each, and applies the pass/fail policy — returning the unsupported clauses so the agentic loop can re-enter targeting exactly what wasn't grounded.

Seam for the reflect grounding/entailment gate: given the clauses of a reflect answer and the text of its cited memories (evidence), return a per-clause verdict on whether the evidence entails the clause.

NLI entailment verifier. Scores each (evidence, clause) pair with a dedicated MNLI cross-encoder — a calibrated, independent entailment signal, the strongest of the three verifier backends, for hosts with the ML headroom (e.g. DGX).

LLM-judge entailment verifier. One structured call over the existing LLM routing decides, for each answer clause, whether the cited-memory text supports it — no new model or serving. Reuses the complete_json_schema/4 seam.

Default entailment verifier: verification OFF. Every clause is reported supported, so the grounding gate is inert until a real verifier is configured — no new cost, latency, or change to the honest baseline unless explicitly opted in.

The agentic reflect expand tool: drill previously retrieved memories up the hierarchy. "chunk" (the default) returns each memory with its provenance and the neighbouring memories of the same document, the surrounding context a truncated fact is missing; "document" returns the owning document's title and text. The id list is model-controlled, so it is de-duplicated, filtered to valid ids and capped before any query runs. cited_ids/1 lists every memory id an expansion put in front of the model, for grounding.

Guards the options reflect hands to LlmCore.Agent.Loop.

Lexical relevance ranking + limit for the reflect search_mental_models tool .

Mental models for reflect, at parity with Hindsight: ranked by embedding similarity to the query, capped, and reported with a per-model staleness verdict.

Builds the text a cross-encoder reranker scores for a candidate.

Retain-time atomic fact extraction — the keystone that gives reflect something structured to aggregate over.

Graded relevance label for a (query, candidate memory) pair — the training/eval substrate for learned per-bank fusion weights.

A directed edge in the memory graph: from_unit_id -> to_unit_id with a link_type and weight, scoped to a bank.

The stored unit of memory: a fact, item, or observation belonging to a bank.

Consolidation orchestration. Phase A (pool existing observations, run the LLM merge, embed results) runs OUTSIDE the DB transaction; Phase B applies the prepared writes, stamps sources, and refreshes mental models INSIDE one transaction. This keeps slow LLM/embedding IO off the tenant DB connection (the reflect txn-across-LLM pool-timeout class of bug).

Consolidation persistence: an Elixir port of Hindsight's engine/consolidation/consolidator.py. Turns raw world/experience facts into consolidated observation memory units via an LLM creates/updates/ deletes merge contract (Foresight.Prompts.Consolidation.V1).

Retain-time construction of the memory_links graph.

The embeddable supervision entry point.

Public tenant-schema provisioning and execution boundaries.

Scrubs model output of characters that break downstream systems.

Faithful port of Python difflib.SequenceMatcher(None, a, b).ratio() — the Ratcliff-Obershelp similarity Hindsight uses for entity name matching (engine/entity_resolver.py:664).

Ledger of the chargeable calls made while serving one operation.

Fetch an evaluation dataset split on demand. Nothing is redistributed in the repo — the data lands in priv/evals/data/, which is gitignored.

Records the FLOOR fixture set from a live hindsight v0.7.1 authority.