Foresight.Retain.FactExtraction (Foresight v0.1.0)

Copy Markdown View Source

Retain-time atomic fact extraction — the keystone that gives reflect something structured to aggregate over.

Faithful port of Hindsight's engine/retain/fact_extraction.py: each incoming retain item (a chat session, document, or paragraph) is decomposed by an LLM into a LIST of atomic facts, one input to N discrete memory units. Whole-session blobs cannot be aggregated across sessions (the buried "took course X" / "spent €40" facts never individuate, so cross-session counts and temporal ordering are unanswerable); atomizing them restores the material that recall ranks, consolidation merges into observations, and reflect sums.

Each produced fact-item INHERITS the parent's provenance and — critically — its mentioned_at (the session date), so every atomic fact carries the timestamp reflect's per-entry temporal arithmetic depends on. Extraction OVERRIDES only content (the assembled fact text), type (world/experience), and the occurred_* dates for datable events.

Extraction is gated: it runs only when a mode is configured AND an extraction-capable LLM is available. It degrades to passthrough (the original item, i.e. today's blob behaviour) on any LLM error, timeout, or empty result — extraction never fails a retain. This mirrors the LlmCore isolation the rest of the pipeline relies on.

Summary

Functions

True when mode and llm would drive real extraction (used by callers to log/branch).

Expand items into atomic fact-items, or return them unchanged (passthrough).

Functions

enabled?(opts)

@spec enabled?(keyword()) :: boolean()

True when mode and llm would drive real extraction (used by callers to log/branch).

maybe_extract(items, opts)

@spec maybe_extract(
  [map()],
  keyword()
) :: [map()]

Expand items into atomic fact-items, or return them unchanged (passthrough).

Opts:

  • :mode — "concise" | "verbatim" | "custom" | nil/"none" (nil/"none" disables)

  • :llm — the Foresight.LLM implementation module (needs complete_json_schema/4)
  • :mission — optional extraction mission steering what is worth keeping
  • :custom_instructions — guidelines body for "custom" mode
  • :provider / :provider_opts / :sleep_ms — passed through to the LLM