Retain-time atomic fact extraction — the keystone that gives reflect something structured to aggregate over.
Faithful port of Hindsight's engine/retain/fact_extraction.py: each incoming
retain item (a chat session, document, or paragraph) is decomposed by an LLM
into a LIST of atomic facts, one input to N discrete memory units. Whole-session
blobs cannot be aggregated across sessions (the buried "took course X" / "spent
€40" facts never individuate, so cross-session counts and temporal ordering are
unanswerable); atomizing them restores the material that recall ranks,
consolidation merges into observations, and reflect sums.
Each produced fact-item INHERITS the parent's provenance and — critically — its
mentioned_at (the session date), so every atomic fact carries the timestamp
reflect's per-entry temporal arithmetic depends on. Extraction OVERRIDES only
content (the assembled fact text), type (world/experience), and the
occurred_* dates for datable events.
Extraction is gated: it runs only when a mode is configured AND an extraction-capable LLM is available. It degrades to passthrough (the original item, i.e. today's blob behaviour) on any LLM error, timeout, or empty result — extraction never fails a retain. This mirrors the LlmCore isolation the rest of the pipeline relies on.
Summary
Functions
True when mode and llm would drive real extraction (used by callers to log/branch).
Expand items into atomic fact-items, or return them unchanged (passthrough).
Functions
True when mode and llm would drive real extraction (used by callers to log/branch).
Expand items into atomic fact-items, or return them unchanged (passthrough).
Opts:
:mode— "concise" | "verbatim" | "custom" | nil/"none" (nil/"none" disables):llm— theForesight.LLMimplementation module (needscomplete_json_schema/4):mission— optional extraction mission steering what is worth keeping:custom_instructions— guidelines body for"custom"mode:provider/:provider_opts/:sleep_ms— passed through to the LLM