barrel_ngram_source behaviour (barrel_ngram v0.9.0)

View Source

Content-source behaviour for windowed verification.

A positional (phase-2) query narrows a match to a small byte window and only needs to read that back, not the whole document. A corpus opened with source => {Module, InitArg} uses this to do that read without barrel_ngram calling into barrel_docdb directly; InitArg is stored once and threaded to every call as State.

Optional: without it, verification fetches full current content via barrel_docdb:get_docs/2 instead. Candidate narrowing itself never depends on source -- only the windowed read does.

pread/4 must return bytes identical to what barrel_ngram_corpus:doc_text/2 produced at index time (a race with a concurrent change may return stale bytes -- accepted, since every read is re-verified before being trusted, so the failure mode is a false negative, never a false positive). EOF/short-read contract, matching file:pread/3:

  • Len =:= 0 always returns {ok, <<>>}, regardless of Offset (as long as Offset >= 0) -- this is what makes an empty document readable at all, e.g. for a pattern that can legitimately match empty content.
  • Len > 0, Offset >= Size -- {error, eof}.
  • Len > 0, Offset < Size, Offset + Len > Size -- {ok, Bin} clamped to the available suffix (byte_size(Bin) =:= Size - Offset), never padded.
  • Len > 0, Offset < Size, Offset + Len =< Size -- {ok, Bin} with byte_size(Bin) =:= Len.
  • Missing or deleted document -- {error, not_found}.

Summary

Functions

Dispatch doc_size/2 to a source module.

Dispatch pread/4 to a source module.

Callbacks

doc_size/2

-callback doc_size(State :: term(), DocId :: binary()) -> {ok, non_neg_integer()} | {error, term()}.

pread/4

-callback pread(State :: term(), DocId :: binary(), Offset :: non_neg_integer(), Len :: non_neg_integer()) ->
                   {ok, binary()} | {error, term()}.

Functions

doc_size(_, DocId)

-spec doc_size({module(), term()}, binary()) -> {ok, non_neg_integer()} | {error, term()}.

Dispatch doc_size/2 to a source module.

pread(_, DocId, Offset, Len)

-spec pread({module(), term()}, binary(), non_neg_integer(), non_neg_integer()) ->
               {ok, binary()} | {error, term()}.

Dispatch pread/4 to a source module.