# Dextrin v0.2.0 - Table of Contents

> DXN (Data eXchange Notation) for Elixir -- .dxn text, .dxnb binary, and .dxns schema documents, built on the Ichor grammar compiler.

## Pages

- [Dextrin](readme.md)

- Guides
  - [Tutorial](tutorial-1.md)
  - [Examples](examples.md)
  - [Cheatsheet](cheatsheet.md)

- DXN Format
  - [DXN Tutorial](tutorial-2.md)
  - [DXN Technical Reference](dxn.md)
  - [DXN Examples](dxn_examples.md)
  - [DXN Cheatsheet](dxn_cheatsheet.md)

- Project
  - [Changelog](changelog.md)
  - [Contributing to Dextrin](contributing.md)
  - [LICENSE](license.md)

## Modules

- [Dextrin.Canonical](Dextrin.Canonical.md): The canonical form of a value (`DXN.md` §1.5): one `.dxn` rendering,
fixed down to the byte, so that values from different implementations
can be compared as text.
- [Dextrin.Conformance](Dextrin.Conformance.md): This implementation's answers to the Dextrin Conformance Suite.
- [Dextrin.Document](Dextrin.Document.md): A `.dxn` document with everything it holds: its value, and every
comment, blank line and comma around it (`design/comment-retention.md`).
- [Dextrin.Document.Formatter](Dextrin.Document.Formatter.md): A document's syntax tree laid out the way `Dextrin.Text.Formatter`
lays out a value -- every non-empty collection over lines, one entry
per line, indented -- with every comment kept beside the entry it
belongs to (`design/comment-retention.md`, "The rules").
- [Dextrin.MapKey](Dextrin.MapKey.md): Which values may be map keys (`DXN.md` §1.6): a `string`, `keyword`,
`integer`, `uuid`, `date`, `bytes`, `symbol` or `char`. Anything else is
refused as `key/not-allowed` -- in text, in binary and when encoding.
- [Dextrin.Schema.Compat](Dextrin.Schema.Compat.md): The version discipline check (DXN.md §4.8): compares the schemas of a
previous release with the current ones.
- [Dextrin.Schema.Reader](Dextrin.Schema.Reader.md): Which schema a decoded struct is read with, given the version it was
written with and the versions loaded (DXN.md §4.8). Shared by the text
and binary decoders, so both answer every case the same way.

- Core
  - [Dextrin](Dextrin.md): Public API for DXN (Data eXchange Notation) — `.dxn` text and
`.dxnb` binary, per `DXN.md` (the normative format specification).
  - [Dextrin.Error](Dextrin.Error.md): One error struct for both the text and binary pipelines, so a caller
never has to special-case which one produced it. Wraps `Ichor.Error`
verbatim for the text side (including its caret-annotated
`context_lines`, useful for CLI/log output); the binary side has no
source text to annotate, so it only ever carries a message, a byte
offset, and a stage.
  - [Dextrin.Registry](Dextrin.Registry.md): Extension point for both `struct` and `custom-tag` decoding/encoding.
No entry for a given name → decode falls back to an opaque
`Dextrin.Struct`/`Dextrin.CustomTag` — never a hard failure, since
neither type requires a registration to be representable at all.
  - [Dextrin.Value](Dextrin.Value.md): The union of every shape a decoded DXN value can take. A typespec
aid only — no functions, no runtime behavior.

- Value types
  - [Dextrin.Array](Dextrin.Array.md): DXN `array` (`@array[ ... ]`) — fixed-size, indexed. Wraps an
Elixir tuple, the one DXN collection type where Elixir's own
fixed-arity tuple is actually the right fit — unlike `Dextrin.Tuple`,
which backs DXN's arbitrary-length `tuple` with a list instead.

  - [Dextrin.Bytes](Dextrin.Bytes.md): DXN `bytes` (`@bytes "..."`, base64 in text, raw binary in `.dxnb`).
Wraps a plain Elixir `binary()` — needed because `string` is *also*
a plain Elixir `binary()` (`String.t()` is not a distinct runtime
type), so a bare binary can't otherwise tell "UTF-8 text" and "raw
bytes that happen to be valid UTF-8" apart on re-encode. The same
kind of ambiguity `Dextrin.Char`/`Dextrin.Symbol`/`Dextrin.Keyword`
each wrap a bare value to avoid.

  - [Dextrin.Char](Dextrin.Char.md): DXN `char` (`?c`) — a single Unicode codepoint. Kept distinct from a
1-grapheme `String.t()` so `char` and `string` (`?a` vs `"a"`, two
distinct DXN types) never collapse to the same Elixir value and
silently fail to round-trip.

  - [Dextrin.CustomTag](Dextrin.CustomTag.md): Opaque DXN custom tag (`@tag value`), produced when no decoder is
registered for its name (`Dextrin.Registry.put_tag/3`) — the open
extension point DXN.md §1.2 reserves for any tag name that isn't one
of the built-ins (`@uuid`, `@duration`, ...). Prefer a schema-backed
`struct` over a custom tag for anything with real field structure;
reserve custom tags for simple scalar-wrapping.

  - [Dextrin.Duration](Dextrin.Duration.md): DXN `duration` (`@duration "P..."`). Each field is
`integer() | nil` — only fields actually present in the source set
a bit in `.dxnb`'s bitmask (DXN.md §2.3). `years`/`months` are
calendar-relative (variable length) and not reducible to a single
elapsed-time scalar, so all seven fields are tracked independently
rather than folded into one (DXN.md §1.4).

  - [Dextrin.Keyword](Dextrin.Keyword.md): DXN `keyword` (`:name` in value position). Wraps a plain
`String.t()`, never an Elixir atom, for the same reason as
`Dextrin.Symbol`: atoms are never garbage collected on the BEAM, and
decoding untrusted, attacker-controlled data must never be able to
exhaust the atom table.

  - [Dextrin.OrderedMap](Dextrin.OrderedMap.md): DXN `ordered-map` (`@ordered %{ ... }`) — order is part of the
value's identity, unlike plain `map`. Wraps an ordered list of
`{key, value}` pairs rather than a plain Elixir map, so insertion
order survives structural `==` comparison.

  - [Dextrin.Rational](Dextrin.Rational.md): DXN `rational` (`int/uint`) — an exact ratio, stored exactly as
given and never silently reduced: `22/7` and `44/14` are distinct DXN
values (`DXN.md` calls it "exact ratio," not "exact reduced ratio"),
so reducing on parse would be a silent, opinionated transformation.
`reduce/1` is offered but never called implicitly.

  - [Dextrin.SortedSet](Dextrin.SortedSet.md): DXN `sorted-set` (`@sorted-set @{ ... }`). Wraps a list kept sorted
(and deduplicated) as a hard invariant at every construction site —
`new/1` is the *only* way to build one, precisely so that structural
`==` between two `Dextrin.SortedSet`s is valid set-equality rather
than something that happens to work only when both were built the
same way — every construction site (decode, encode, public API)
goes through `new/1`, so the sorted/deduplicated invariant can never
be bypassed.

  - [Dextrin.Struct](Dextrin.Struct.md): Opaque DXN `struct` value, produced when no schema is compiled for
its name (DXN.md §1.4: "a reader lacking the schema... returns an
opaque tagged value rather than failing").
  - [Dextrin.Symbol](Dextrin.Symbol.md): DXN `symbol` — a bare, unevaluated identifier reference. Wraps a
plain `String.t()`, never an Elixir atom: atoms are never garbage
collected on the BEAM, and a decoder fed adversarial or merely large
third-party input must not be able to exhaust the atom table by
decoding enough distinct symbols. A caller who trusts their input
and wants a real atom can always convert explicitly.

  - [Dextrin.Tuple](Dextrin.Tuple.md): DXN `tuple` (`{ ... }`) — ordered, heterogeneous, arbitrary length.
Wraps a list, not an Elixir tuple: DXN tuples have no fixed arity in
the type system the way Elixir's do, and `Dextrin.Array` already
claims the Elixir tuple for the one DXN type that actually is
fixed-size/indexed.

  - [Dextrin.Uri](Dextrin.Uri.md): DXN `uri` (`@uri "..."`, RFC 3986). Wraps the raw string exactly as
given, not parsed into `URI.t()` — `URI.parse/1` is lenient in ways
RFC 3986 isn't, and `URI.to_string/1` doesn't always reproduce the
original text byte-for-byte (e.g. component re-escaping), which would
risk breaking round-trip. Parsing is left to the caller, who can
always call `URI.parse/1` on `value` themselves.

  - [Dextrin.Uuid](Dextrin.Uuid.md): DXN `uuid` (`@uuid "..."`, RFC 4122). Wraps 16 raw bytes, not the
36-char text form — the text form exists only at the `.dxn`/`.dxnb`
encoding boundary (`parse/1`/`format/1` below convert between them);
the in-memory value stays the compact, comparison-cheap byte form.

- Text pipeline (.dxn)
  - [Dextrin.Text.Actions](Dextrin.Text.Actions.md): `Ichor.Actions` implementation for `Dextrin.Text.Grammar` — turns
the raw capture tree `Grammar.Native` produces while matching
`priv/grammar/dxn.aether` into actual `Dextrin.Value.t()` values.
`context` is a `Dextrin.Registry.t()` — text parsing only ever reads
it (to resolve struct schemas and custom tags), never mutates it.
  - [Dextrin.Text.Escapes](Dextrin.Text.Escapes.md): Shared escape-decoding for DXN's `string`/`char`/quoted-`keyword`
bodies (`DXN.md` §1.1's `escape` production) — one implementation so
none of the token handlers in `Dextrin.Text.Actions` duplicate it.

  - [Dextrin.Text.Formatter](Dextrin.Text.Formatter.md): Multi-line, indented `.dxn` rendering — `Dextrin.encode/2`'s
`pretty: true` opt, `mix dextrin.format`'s `--mode pretty`, and `mix
dextrin.decode`'s default output (a CLI decode is a human reading
the result, so `encode/2`'s own single-line default isn't the right
default there). `--mode condense` needs no separate implementation
at all: it's exactly what `Dextrin.encode/2`/`Dextrin.Text.Printer`
already produce with `pretty: false` (the default).
  - [Dextrin.Text.Grammar](Dextrin.Text.Grammar.md): Compiles `priv/grammar/dxn.aether` via Ichor's `Grammar.Native`
backend, not `Grammar.VM`'s interpreted backend — the grammar is
fixed at `dextrin`'s own build time (there's no scenario where a
caller supplies a different grammar at runtime), so there's no
reason to pay for bytecode interpretation when a compiled parser is
available for free.
  - [Dextrin.Text.Printer](Dextrin.Text.Printer.md): `.dxn` printer — the reverse of `Dextrin.Text.Actions`. Single-line,
minimal-whitespace output: a printer, not a formatter (no
line-wrapping or indentation policy is specified anywhere in
`DXN.md`) — `Dextrin.Text.Formatter.pretty/2` owns multi-line,
human-readable rendering, built on top of this module.

- Binary pipeline (.dxnb)
  - [Dextrin.Binary.Decoder](Dextrin.Binary.Decoder.md): `.dxnb` decoder — the mirror image of `Dextrin.Binary.Encoder`.
Envelope (`magic` `version` `cbor_item`) checked once at the top;
everything below that is a single recursive `decode_item/2`, keyed
on CBOR major type first, then on tag — matching `DXN.md` §2.2's
table row order the same way the encoder does.

  - [Dextrin.Binary.Encoder](Dextrin.Binary.Encoder.md): `.dxnb` encoder — a recursive walk over CBOR's major types plus a
fixed tag dispatch table, matching `DXN.md` §2.2's table row order.
  - [Dextrin.Binary.Tags](Dextrin.Binary.Tags.md): Pure constants — CBOR tag numbers and bit-layout tables, mirroring
`DXN.md` §2.3 verbatim. No behavior lives here.

- Schema (.dxns)
  - [Dextrin.Schema](Dextrin.Schema.md): Public entry point for `.dxns` schema compilation and validation.
  - [Dextrin.Schema.Compiled](Dextrin.Schema.Compiled.md): One version of one schema, compiled from a `%schema(N){...}` entry of a
`.dxns` document (`Dextrin.Schema.Compiler.compile/2`) -- enough to both
validate a decoded value and convert it both directions between
`.dxnb`'s always-positional wire shape and a named field map.
  - [Dextrin.Schema.Compiler](Dextrin.Schema.Compiler.md): Compiles a parsed `.dxns` document (an ordinary decoded DXN value —
a `.dxns` file is valid `.dxn`, no new grammar; what makes it a
*schema* document is purely the shape of the value it parses to: a
map from name to type expression) into `Dextrin.Schema.Compiled`
entries.
  - [Dextrin.Schema.Field](Dextrin.Schema.Field.md): One compiled field spec inside a `Dextrin.Schema.Compiled` struct.
`required` comes from the `?`-suffixed key convention (a field key
ending in `?` is optional; no separate flag exists in `.dxns` itself
— reusing DXN's own identifier grammar rather than adding one);
`default`/`description` only ever come from the `%field{...}` escape
hatch, since optionality is already fully covered by the key suffix.

  - [Dextrin.Schema.FileResolver](Dextrin.Schema.FileResolver.md): An optional convenience `struct_resolver` resolving `Namespace/Name`
references to `.dxns` files on disk.
  - [Dextrin.Schema.Provider](Dextrin.Schema.Provider.md): Lets a struct's *own* library ship a DXN schema for it — field
names, types, required/optional, refinements, a materializer —
without that library ever taking a hard dependency on `dextrin`. This is the `struct` (schema-backed) counterpart to what
`Dextrin.Registry.put_tag_encoder/4` already does for simple
scalar-wrapping `custom-tag` values: an extension point owned by the
*consuming* application, not forced onto every struct-defining
library it might want to use with `dextrin`.
  - [Dextrin.Schema.Std](Dextrin.Schema.Std.md): dextrin's standard library of named types (`priv/schema/std.dxns`)
— common refinements like `PositiveInteger` or `NonEmptyString`, so
a schema author doesn't redefine them by hand. Built the same way
any consumer's own named types would be — nothing about them is
special-cased in `Dextrin.Schema.Compiler`. Deliberately excludes
anything domain-specific (email, phone number, URL-shaped string):
what counts as a valid one is an application decision this library
shouldn't guess at.
Opt-in: pass `Std.registry/1`'s result as `compile/3`'s
`base_registry` to make these names available to your own document.
  - [Dextrin.Schema.TypeExpr](Dextrin.Schema.TypeExpr.md): Internal, compiled representation of a `.dxns` type expression — the
13-form vocabulary (`any`, `primitive`, `reference`, `list-of`,
`set-of`, `tuple-of`, `map-of`, `enum`, `one-of`, `all-of`,
`nilable`, `refine`, `struct`) `Dextrin.Schema.Compiler` turns a
parsed `.dxns` value into, and what `matches?/2` below checks a
decoded value against. This vocabulary is fixed, not extensible from
outside an actual library change — what schema *authors* extend
instead is composing these forms into a reusable named type (see
`Dextrin.Schema.Compiler`'s own moduledoc), which needs no new form
at all.

  - [Dextrin.Schema.Validated](Dextrin.Schema.Validated.md): Internal only — never appears in a value handed back to
`Dextrin.decode/2`/`decode_binary/2`'s caller.
  - [Dextrin.Schema.Validator](Dextrin.Schema.Validator.md): Checks a value's fields against a `Dextrin.Schema.Compiled` schema
(required fields and declared types) — shared by decode (`resolve/3`,
whose result `Dextrin.Schema.Reader` builds a value from) and encode-time validation
(`validate_for_encode/3`/`validate_tree_for_encode/2`, which never
transforms `value`). Both directions reuse the exact same
`resolve_fields/3`/`TypeExpr.matches?/3` field-checking — encode's
own values are wrapped in the same internal `Dextrin.Schema.
Validated` marker decode already uses, via `wrap_and_check/2`, so
there's one type-checking implementation, not two that could drift.

- Unicode
  - [Dextrin.Unicode.RangeGenerator](Dextrin.Unicode.RangeGenerator.md): Pure text-processing core for turning a Unicode Character Database
`DerivedCoreProperties.txt` into the generated `IDENT_START`/
`IDENT_CONT`/`IDENTIFIER` block spliced into `priv/grammar/dxn.aether`.
No file or network I/O lives here — that's
`Mix.Tasks.Dextrin.Gen.Unicode`'s job — so this module can be tested
directly against small in-memory fixtures instead of the real,
~1MB UCD file.

## Mix Tasks

- [mix dextrin.compat](Mix.Tasks.Dextrin.Compat.md): The version discipline check (`Dextrin.Schema.Compat`, DXN.md §4.8):
compares the schemas in `OLD` with those in `NEW`, each a `.dxns` file
or a directory searched for them. A version whose content changed
without a new version, or a version that went down, is reported and
exits non-zero, so CI can run it against the previous release's schema
files. What the latest version's materializer has to upgrade is listed
either way.
- [mix dextrin.conformance](Mix.Tasks.Dextrin.Conformance.md):     $ mix dextrin.conformance REQUEST ANSWER

- Mix tasks
  - [mix dextrin.decode](Mix.Tasks.Dextrin.Decode.md):     $ mix dextrin.decode data.dxnb
    $ mix dextrin.decode data.dxnb --out data.dxn
  - [mix dextrin.encode](Mix.Tasks.Dextrin.Encode.md):     $ mix dextrin.encode data.dxn
    $ mix dextrin.encode data.dxn --out data.dxnb
    $ mix dextrin.encode data.dxn --share
  - [mix dextrin.format](Mix.Tasks.Dextrin.Format.md):     $ mix dextrin.format data.dxn --mode pretty
    $ mix dextrin.format data.dxn --mode condense
    $ mix dextrin.format data.dxn --mode pretty --in-place
  - [mix dextrin.gen.schema](Mix.Tasks.Dextrin.Gen.Schema.md):     $ mix dextrin.gen.schema MyApp.Point
    $ mix dextrin.gen.schema MyApp.Point --out point.dxns --name Point
  - [mix dextrin.gen.unicode](Mix.Tasks.Dextrin.Gen.Unicode.md): Fetches the latest Unicode Character Database `DerivedCoreProperties.txt`,
compares its version against the one last processed
(`priv/unicode/VERSION`), and — if newer — regenerates the
`IDENT_START`/`IDENT_CONT`/`IDENTIFIER` ranges spliced into
`priv/grammar/dxn.aether` (see `Dextrin.Unicode.RangeGenerator` for
the pure text-processing logic this task wraps with file/network I/O).
  - [mix dextrin.validate](Mix.Tasks.Dextrin.Validate.md): Decodes `PATH` (format sniffed from its extension, or forced with
`--format text|binary`), reporting success or a rendered
`Dextrin.Error` — non-zero exit on failure, so this is meant for CI
as much as interactive use.

