barrel_ngram_selector behaviour (barrel_ngram v0.7.1)
View SourceGram-selection behaviour: the shared seam between the indexer and the query planner.
A selector maps a byte string to the set of trigrams it contributes to the index. The SAME selector is applied by the indexer (over full document bytes) and by the query planner (over the query literal), so that the grams a literal produces are always a subset of the grams its containing documents produced. That subset relationship is what makes the trigram intersection a correct necessary-condition filter.
Two callbacks, each taking the bytes and a selector-options map:
select_grams/2- the grams a byte string contributes to the index. Used at index time.reliable_grams/2- the grams of a query literal the planner may safely intersect over, orbrute_forcewhen it may not (too short, or every gram sits on an unreliable boundary). Used at query time.
For the dense selector every gram is reliable, so reliable_grams/2 returns all of them (or brute_force below the trigram length). The boundary/interior distinction bites the content-defined (sparse) selector, which lives behind this same behaviour without changing anything above it. The options map carries per-selector tuning (e.g. the sparse selector's window radius and sample rate).
Summary
Functions
Whether the selector indexes every trigram (dispatch).
Dispatch reliable_grams/2 to a selector module.
Dispatch select_grams/2 to a selector module.
Types
-type gram() :: 0..16777215.
-type reliable() :: {reliable, [gram()]} | brute_force.
Callbacks
Functions
Whether the selector indexes every trigram (dispatch).
Dispatch reliable_grams/2 to a selector module.
Dispatch select_grams/2 to a selector module.