Pure text-processing core for turning a Unicode Character Database
DerivedCoreProperties.txt into the generated IDENT_START/
IDENT_CONT/IDENTIFIER block spliced into priv/grammar/dxn.aether.
No file or network I/O lives here — that's
Mix.Tasks.Dextrin.Gen.Unicode's job — so this module can be tested
directly against small in-memory fixtures instead of the real,
~1MB UCD file.
DXN.md's letter/ident_char productions are defined via
Unicode's XID_Start/XID_Continue properties (UAX #31) — hundreds
of codepoint ranges, not something any human should hand-write or
hand-maintain across Unicode versions. This module turns the UCD's
own published range data into Aether character-class syntax
(\u{H+} escapes), so keeping the grammar current is "run the mix
task, review the diff," not manual transcription.
Summary
Functions
Renders the full generated grammar block (marker comments included) for a given UCD text.
Counts of merged XID_Start/XID_Continue ranges — for status/logging output.
Extracts and merges the codepoint ranges for one named derived property (e.g. "XID_Start").
Splices generated into grammar_source, replacing the existing
marked region if one exists, or appending one if not.
Extracts the Unicode version from a DerivedCoreProperties.txt's own header line, e.g. "17.0.0".
Functions
Renders the full generated grammar block (marker comments included) for a given UCD text.
@spec range_counts(String.t()) :: {non_neg_integer(), non_neg_integer()}
Counts of merged XID_Start/XID_Continue ranges — for status/logging output.
@spec ranges(String.t(), String.t()) :: [{non_neg_integer(), non_neg_integer()}]
Extracts and merges the codepoint ranges for one named derived property (e.g. "XID_Start").
Splices generated into grammar_source, replacing the existing
marked region if one exists, or appending one if not.
Extracts the Unicode version from a DerivedCoreProperties.txt's own header line, e.g. "17.0.0".