How recall works

Recall runs four searches at once and merges them into one ranked list. The merge, it turns out, matters more than most of the searches.

Four ways of being relevant

A memory can be worth returning for quite different reasons, and no single index captures all of them.

Semantic — embedding similarity. Finds "the deploy took ages" when you asked about "slow migrations". Blind to exact identifiers: it has no special respect for v2.3. A candidate has to clear a cosine-similarity floor to enter this arm at all: the bank config field recall_semantic_floor, default 0.3. The right value depends on the embedding model. BAAI/bge-small-en-v1.5 scores unrelated text at roughly 0.33–0.36 and related text at 0.55–0.78, so on that model the default admits nearly the whole bank and leaves the reranker to sort it out, while about 0.45 already sheds the off-topic candidates. Measure on your own corpus before raising it; a floor above the model's related-text band empties the arm. The trace reports the floor in force as semantic_similarity_floor.

Keyword — Postgres full-text search over a stored tsvector. The opposite temperament. It will find v2.3 and KB-4156 precisely, and miss every paraphrase. Deliberately faithful to Hindsight here: tokens are OR-joined with no :* prefix matching, so run does not silently match running.

Graph — traversal of the entity graph built during retain. Finds memories that never mention your query terms but are about an entity that does. This is the arm that connects "Maria" to "Lisbon" without either sentence containing both.

Temporal — time-proximity. Matters for "what else was happening that week?". Worth knowing: on our benchmark corpus this arm fires on 29 of 4,785 candidates. It is a specialist, not a workhorse, and any tuning that appears to learn something about it is almost certainly learning noise.

Each arm returns its own ranked list with its own scores, on its own scale.

The merge is the interesting part

Four lists have to become one. The obvious approach — the one Foresight shipped originally and Hindsight still uses — is Reciprocal Rank Fusion: ignore the scores, use only positions, sum 1/(k + rank) across arms.

RRF is robust precisely because it discards the scores. It cannot be fooled by one arm's scale.

It can, however, be blinded by it. Consider semantic returning:

1. 0.95  "the payments migration slipped to Q3"
2. 0.31  "payments team standup notes"

To RRF these are simply first and second. The chasm between 0.95 and 0.31 — information the embedder computed and handed over — is thrown away.

Normalizing instead keeps it: rescale each arm's native scores to a common range, then add. We measured both over 134 queries carrying graded relevance labels, paired per query, with a 2,000-sample bootstrap:

zscore beats min_max because a single outlier squashes a min-max range, while a standard-deviation scale is unmoved by it.

:normalized with :zscore is the default. The number to watch is MRR 0.776 → 0.903: that is the right memory sitting at rank 1 far more often, and rank 1 is what reflect reads first.

What that result does not say

It is one corpus, and the relevance labels were produced by a model rather than a person. A ranking improvement is not an answer-quality improvement — the same change did not produce a significant lead in end-to-end accuracy. See Evaluation.

Weights, and a negative result worth reading

:normalized and :hybrid accept per-arm weights:

config :foresight,
  recall_fusion: [
    mode: :normalized,
    normalization: :zscore,
    weights: %{"semantic" => 1.5, "keyword" => 1.0, "graph" => 0.5, "temporal" => 1.0}
  ]

Weights are ignored under :rrf, which ranks on position alone.

We tried to learn these weights per bank — coordinate ascent on a training split, scored once on held-out queries. Across 12 query-level splits it was worth +0.0153 nDCG@10 (SD 0.0275), and negative in 3 of them. Not significant, and not shipped.

The honest lesson is that picking the right merge strategy was worth ~7× more than tuning weights within it, and cost nothing at runtime. If you are tempted to tune weights, measure first — and measure on held-out queries, not the ones you tuned on.

After the merge

The fused list is truncated to a rerank window, optionally passed to a cross-encoder reranker, boosted, and trimmed. The default reranker is Passthrough — it preserves fusion order — so out of the box your ranking is the fusion result. Configure Rerankers.Bumblebee or Rerankers.SentenceTransformers for real reranking, at the cost of the ML stack.

The rerank window

The window is twice the resolved recall budget (Hindsight parity: budget low is 100, so 200 candidates go to the cross-encoder), or the bank config field recall_rerank_window when set. On a bank with a few hundred fused candidates per query that means all of them are reranked, and the cross-encoder's cost is linear in the window. cross-encoder/ms-marco-MiniLM-L-6-v2 scores ~200 pairs in about a second on an Apple GPU; BAAI/bge-reranker-v2-m3 takes five to eight times longer on the same passages. If you run a heavy reranker, set recall_rerank_window to something like 40–100 and measure what it costs you in hit@k before going wider — every response carries the numbers to do that (see stats below).

Which reranker

The default sentence-transformers model, MS MARCO MiniLM, is tuned for web passages and flattens abstract or philosophical paraphrase: it scores a correct paraphrase and an off-topic sentence at the same logit (around −11). On a labelled 20-query set over such a corpus, BAAI/bge-reranker-v2-m3 moved hit@1 from 13/20 to 18/20 and MRR from 0.77 to 0.93, for the latency above. Pick the model for the corpus, set FORESIGHT_RERANKER_MODEL_ID (or :ml, :reranker_model_id), and re-measure any min_score cutoff: different models put their logits on different scales.

The reranker never holds a database connection

Under Mode C tenancy a scope is a checked-out connection with SET ROLE, and the pool's checkout timeout (15 s by default) counts everything run inside it. Recall therefore runs as scoped database phases — bank config, retrieval, result enrichment — with the query embedding and the reranker between them, on no connection at all. A reranker that takes ten seconds makes a slow recall, not a dropped connection and a failed one.

The score, and cutting off on it

Every result carries "score": the reranker's calibrated score after boosts, the number the results are ordered by. With the sentence-transformers reranker the pre-boost value is sigmoid(cross-encoder logit): the worker always hands back raw logits, whatever the model's own default activation. Different models put their logits on different scales, so re-measure any cutoff when you change the model. Pass "min_score" (a non-negative number) to drop everything below it; the cutoff runs after boosting and before the token trim, so the trim never spends budget on results you would discard. Both are Foresight extensions to the Hindsight recall contract.

{:ok, %{"results" => results}} =
  Foresight.recall(ctx, %{"query" => "...", "min_score" => 0.5})

Enum.map(results, & &1["score"])  #=> [0.91, 0.74, 0.52]

Seeing what happened

Every response carries a small stats map: how many fused candidates there were, the rerank window in force, how many candidates were actually reranked, how many results survived, and the retrieval, rerank and total milliseconds. It is cheap, always on, and the first thing to look at when recall is slow:

{:ok, %{"stats" => stats}} = Foresight.recall(ctx, %{"query" => "..."})
stats
#=> %{"candidates" => 187, "rerank_window" => 200, "rerank_candidates" => 187,
#     "results" => 12, "retrieval_ms" => 410, "rerank_ms" => 6_900, "total_ms" => 7_400}

That shape says the reranker is doing all the work over the whole fused set; a smaller recall_rerank_window is the first lever.

Pass trace: true and recall explains itself in full: per-arm entry points and native scores, the fused ordering, what each stage pruned, and per-stage timings_ms.

{:ok, %{"trace" => trace}} = Foresight.recall(ctx, %{"query" => "...", "trace" => true})
trace["fusion"]  #=> %{"algorithm" => "normalized", "normalization" => "zscore", "k" => 60}

That "algorithm" field is derived from the configuration actually in force. It used to be hardcoded to "rrf", which meant that after the default changed it confidently reported the wrong algorithm on every single request — the one field an operator would consult to find out what really ran.