Discovers near-duplicate semantic nodes through the tag-induced projection (nodes sharing at least one tag) and merges them via an LLM that synthesizes a combined statement.
For each semantic node, the most similar tag-neighbor above the threshold is considered for merging. The LLM classifies the pair: same-fact updates and well-merging topics are merged into the higher-scored survivor, whose proposition is replaced by the merged statement and re-embedded, while the other node's links and metadata transfer to the survivor before it is deleted. Weakly related pairs are kept separate to avoid stitching unrelated facts together. Cleans up orphaned tags after consolidation.
Summary
Functions
Finds near-duplicate semantic nodes via their shared tags and merges them.
Functions
@spec consolidate(keyword()) :: {:ok, %{ deleted: non_neg_integer(), checked: non_neg_integer(), merged: non_neg_integer(), deleted_ids: [String.t()] }, {module(), term()}} | {:error, term()}
Finds near-duplicate semantic nodes via their shared tags and merges them.
For each semantic node, candidates are the semantic nodes attached to at least one of its tags. The most similar candidate above the threshold is submitted to the LLM, which either synthesizes a merged statement (the pair is consolidated into the higher-scored node) or rejects the merge (both nodes are kept).
Options
:backend-{module, state}tuple (required):config-%Mnemosyne.Config{}(required):llm- LLM adapter module (required):embedding- embedding adapter module (required):threshold- cosine similarity above which a pair is submitted for merging (default0.7)
Returns {:ok, %{deleted: n, checked: n, merged: n, deleted_ids: [id]}, {backend_mod, new_state}}.