Mnemosyne.Pipeline.SemanticConsolidator (mnemosyne v0.1.6)

Copy Markdown View Source

Discovers near-duplicate semantic nodes via embedding similarity and merges the lower-scored one into the higher-scored survivor.

Transfers all graph connections (tag memberships, sibling links, provenance links) from the loser to the winner and merges metadata. Cleans up orphaned tags after consolidation.

Summary

Functions

consolidate(opts)

Finds near-duplicate semantic nodes and merges them.

Functions

consolidate(opts)

@spec consolidate(keyword()) ::
  {:ok,
   %{
     deleted: non_neg_integer(),
     checked: non_neg_integer(),
     deleted_ids: [String.t()]
   }, {module(), term()}}
  | {:error, term()}

Finds near-duplicate semantic nodes and merges them.

Performs pairwise embedding comparison across all semantic nodes, transfers the loser's links and metadata to the winner, then deletes losers and any orphaned tags.

Options

  • :backend - {module, state} tuple (required)
  • :config - %Mnemosyne.Config{} (required)
  • :threshold - cosine similarity above which nodes are duplicates (default 0.85)

Returns {:ok, %{deleted: n, checked: n, deleted_ids: [id]}, {backend_mod, new_state}}.