barrel_ngram_planner (barrel_ngram v0.9.0)
View SourceQuery planner: decides, per segment, how much a literal's or regex's phase-2 (positional) postings can narrow candidates.
literal_plan/2 picks a literal's reliable (interior) grams once; segment_plan/2 ranks them by this segment's own doc-count and picks the cheapest one or two to distance-check -- which ones only affects speed, never correctness. A plan survivor is a candidate match start, not a confirmed one; verification (see barrel_ngram_verify) always follows.
case_mode/1/regex_case_mode/2 skip phase-2 for a case-insensitive query (its sampling is itself case-sensitive): an ASCII-only literal/pattern narrows via case-variant expansion and verifies [caseless]; any non-ASCII byte narrows not at all (all) and verifies [caseless, unicode]. Verification always compiles escape_literal/1's output, never the raw literal -- re:compile rejects [literal, caseless] together.
regex_plan/2 is the regex analog: full_scan for anything not a clean, anchor-free literal-run chain with a bounded, reliably-sampled run (see barrel_ngram_regex:literal_runs/1); otherwise {windowed, AnchorBytes, PrefixMax, SuffixMax, GramOffs} on the longest such run.
Summary
Functions
The ASCII case-insensitive narrowing query: OR each trigram position's case variants, AND across positions. all below a trigram.
The phase-1 narrowing query and re verification options for a case-insensitive literal (see the moduledoc). {error, {invalid_literal_encoding, Literal}} for a non-ASCII, non-UTF-8 literal.
Literal with every PCRE metacharacter escaped, so compiling it as an ordinary pattern matches the literal bytes and nothing else. Byte-level, safe for UTF-8: a continuation/lead byte is always >= 16#80.
A literal's reliable (interior) phase-2 grams, each with its own offset -- computed once, reused across every segment. brute_force below 3 + 2*radius bytes (no interior position possible).
The re compile options for a case-insensitive regex -- same ASCII/non-ASCII split as case_mode/1, but never a narrowing query (always phase-1 all). HasLeadingCaseless: the pattern's own leading (?i) already implies caseless, but unicode still needs adding explicitly for a non-ASCII pattern.
Whether a successfully-barrel_ngram_regex:analyze/1'd regex can anchor a window, and on which literal run -- see the moduledoc.
This segment's best narrowing for a literal's reliable grams: dense if none have phase-2 data here, a single-gram candidate list for one, distance-checked pair for two or more.
Whether Bin is valid UTF-8, via a round-trip through unicode:characters_to_binary/1 (valid input comes back unchanged).
Types
-type case_mode() :: {barrel_ngram_regex:query(), re_opts(), ValidateDocs :: boolean()} | {error, {invalid_literal_encoding, binary()}}.
-type gram() :: barrel_ngram_selector:gram().
-type offset() :: barrel_ngram_postings_positional:offset().
-type ordinal() :: barrel_ngram_postings:ordinal().
-type re_opts() :: [caseless | unicode].
-type regex_plan() :: full_scan | {windowed, binary(), non_neg_integer(), non_neg_integer(), [{gram(), offset()}]}.
Functions
-spec ascii_caseless_query(binary()) -> barrel_ngram_regex:query().
The ASCII case-insensitive narrowing query: OR each trigram position's case variants, AND across positions. all below a trigram.
The phase-1 narrowing query and re verification options for a case-insensitive literal (see the moduledoc). {error, {invalid_literal_encoding, Literal}} for a non-ASCII, non-UTF-8 literal.
Literal with every PCRE metacharacter escaped, so compiling it as an ordinary pattern matches the literal bytes and nothing else. Byte-level, safe for UTF-8: a continuation/lead byte is always >= 16#80.
A literal's reliable (interior) phase-2 grams, each with its own offset -- computed once, reused across every segment. brute_force below 3 + 2*radius bytes (no interior position possible).
-spec regex_case_mode(binary(), boolean()) -> {re_opts(), ValidateDocs :: boolean()} | {error, {invalid_literal_encoding, binary()}}.
The re compile options for a case-insensitive regex -- same ASCII/non-ASCII split as case_mode/1, but never a narrowing query (always phase-1 all). HasLeadingCaseless: the pattern's own leading (?i) already implies caseless, but unicode still needs adding explicitly for a non-ASCII pattern.
-spec regex_plan(term(), map()) -> regex_plan().
Whether a successfully-barrel_ngram_regex:analyze/1'd regex can anchor a window, and on which literal run -- see the moduledoc.
-spec segment_plan(barrel_ngram_segment:handle(), brute_force | {reliable, [{gram(), offset()}]}) -> segment_plan().
This segment's best narrowing for a literal's reliable grams: dense if none have phase-2 data here, a single-gram candidate list for one, distance-checked pair for two or more.
Whether Bin is valid UTF-8, via a round-trip through unicode:characters_to_binary/1 (valid input comes back unchanged).