Table-driven engine implementing UAX #14 line breaking, with CLDR's CJK locale tailoring.
Generated from the LineBreak state machine tables published in PRI #555. See
Unicode.String.Dfa for the engine and Unicode.String.Break.Tailoring for
the locale tailoring.
The arity-1 functions implement UAX #14 as published. The arity-2 and arity-3 functions add the locale tailoring on top.
Summary
Functions
Returns true when a segment boundary falls between string_before and
string_after.
Returns true when a line break opportunity falls between the two strings.
Returns {segment, rest}, or nil when string is empty.
Returns {segment, rest}, or nil when string is empty.
Splits string using the rules alone, without the dictionary pass.
Splits string into segments.
Splits string into line-break segments under locale.
Lazily splits string into line-break segments under locale.
Functions
Returns true when a segment boundary falls between string_before and
string_after.
The automaton always restarts at a boundary, so the question "is there a break at this join" is answered by segmenting from the start of the combined text and asking whether any boundary lands exactly on the join.
Returns true when a line break opportunity falls between the two strings.
Returns {segment, rest}, or nil when string is empty.
Returns {segment, rest}, or nil when string is empty.
Splits string using the rules alone, without the dictionary pass.
Splits string into segments.
Where the data defines dictionary symbols, the rule-based segments are passed through the dictionary breaker afterwards, which is how the standard describes complex context-dependent breaking being triggered.
Splits string into line-break segments under locale.
The tailoring is applied once for the whole string rather than once per segment, so a tailored locale costs one extra pass over the input rather than one per boundary.
Lazily splits string into line-break segments under locale.
The stream carries the tailored text alongside the original and the offset reached in it, so the tailoring is applied once at construction and every segment is still sliced from the original string.