barrel_record_backfill (barrel v1.0.1)

View Source

Vector backfill for a freshly-branched record database.

A branch forks the docdb instantly but starts with an EMPTY vector store. This walks every current document once (chunked changes feed) and rebuilds the index: documents carrying an _embedding (client or computed) index directly with zero embedder calls (stored computed vectors are trusted: a stale one implies a pending outbox entry, which the fork copied and the branch's own indexer heals); documents without one whose policy matches embed their policy text (batch, per-item fallback) and write the vector back; everything else is skipped. Documents rewound by PITR lost their embedding column (archived bodies do not carry it), so the embed leg re-derives their vectors from the restored text.

Runs synchronously in barrel:branch/3; a vector-store write error aborts (the caller tears the branch down), per-document embed failures only count in the report.

Summary

Types

report/0

-type report() ::
          #{indexed := non_neg_integer(),
            embedded := non_neg_integer(),
            skipped := non_neg_integer(),
            failed := non_neg_integer()}.

Functions

run(Opts)

-spec run(#{db := binary(),
            vstore := atom() | binary(),
            policy := barrel_embedding_policy:policy(),
            embed := term(),
            dimensions := pos_integer(),
            batch_size => pos_integer()}) ->
             {ok, report()} | {error, term()}.