# Dextrin v0.1.0 - Table of Contents

> DXN (Data eXchange Notation) for Elixir -- .dxn text, .dxnb binary, and .dxns schema documents, built on the Ichor grammar compiler.

## Pages

- [Dextrin](readme.md)

- Guides
  - [Tutorial](tutorial-1.md)
  - [Examples](examples.md)
  - [Cheatsheet](cheatsheet.md)

- DXN Format
  - [DXN Tutorial](tutorial-2.md)
  - [DXN Technical Reference](dxn.md)
  - [DXN Examples](dxn_examples.md)
  - [DXN Cheatsheet](dxn_cheatsheet.md)

- Project
  - [Changelog](changelog.md)
  - [Contributing to Dextrin](contribution.md)
  - [LICENSE](license.md)

## Modules

- Core
  - [Dextrin](Dextrin.md): Public API for DXN (Data eXchange Notation) — `.dxn` text and
`.dxnb` binary, per `DXN.md` (the normative format specification).
  - [Dextrin.Error](Dextrin.Error.md): One error struct for both the text and binary pipelines, so a caller
never has to special-case which one produced it. Wraps `Ichor.Error`
verbatim for the text side (including its caret-annotated
`context_lines`, useful for CLI/log output); the binary side has no
source text to annotate, so it only ever carries a message, a byte
offset, and a stage.
  - [Dextrin.Registry](Dextrin.Registry.md): Extension point for both `struct` and `custom-tag` decoding/encoding.
No entry for a given name → decode falls back to an opaque
`Dextrin.Struct`/`Dextrin.CustomTag` — never a hard failure, since
neither type requires a registration to be representable at all.
  - [Dextrin.Value](Dextrin.Value.md): The union of every shape a decoded DXN value can take. A typespec
aid only — no functions, no runtime behavior.

- Value types
  - [Dextrin.Array](Dextrin.Array.md): DXN `array` (`@array[ ... ]`) — fixed-size, indexed. Wraps an
Elixir tuple, the one DXN collection type where Elixir's own
fixed-arity tuple is actually the right fit — unlike `Dextrin.Tuple`,
which backs DXN's arbitrary-length `tuple` with a list instead.

  - [Dextrin.Bytes](Dextrin.Bytes.md): DXN `bytes` (`@bytes "..."`, base64 in text, raw binary in `.dxnb`).
Wraps a plain Elixir `binary()` — needed because `string` is *also*
a plain Elixir `binary()` (`String.t()` is not a distinct runtime
type), so a bare binary can't otherwise tell "UTF-8 text" and "raw
bytes that happen to be valid UTF-8" apart on re-encode. The same
kind of ambiguity `Dextrin.Char`/`Dextrin.Symbol`/`Dextrin.Keyword`
each wrap a bare value to avoid.

  - [Dextrin.Char](Dextrin.Char.md): DXN `char` (`?c`) — a single Unicode codepoint. Kept distinct from a
1-grapheme `String.t()` so `char` and `string` (`?a` vs `"a"`, two
distinct DXN types) never collapse to the same Elixir value and
silently fail to round-trip.

  - [Dextrin.CustomTag](Dextrin.CustomTag.md): Opaque DXN custom tag (`@tag value`), produced when no decoder is
registered for its name (`Dextrin.Registry.put_tag/3`) — the open
extension point DXN.md §1.2 reserves for any tag name that isn't one
of the built-ins (`@uuid`, `@duration`, ...). Prefer a schema-backed
`struct` over a custom tag for anything with real field structure;
reserve custom tags for simple scalar-wrapping.

  - [Dextrin.Duration](Dextrin.Duration.md): DXN `duration` (`@duration "P..."`). Each field is
`integer() | nil` — only fields actually present in the source set
a bit in `.dxnb`'s bitmask (DXN.md §2.3). `years`/`months` are
calendar-relative (variable length) and not reducible to a single
elapsed-time scalar, so all seven fields are tracked independently
rather than folded into one (DXN.md §1.4).

  - [Dextrin.Keyword](Dextrin.Keyword.md): DXN `keyword` (`:name` in value position). Wraps a plain
`String.t()`, never an Elixir atom, for the same reason as
`Dextrin.Symbol`: atoms are never garbage collected on the BEAM, and
decoding untrusted, attacker-controlled data must never be able to
exhaust the atom table.

  - [Dextrin.OrderedMap](Dextrin.OrderedMap.md): DXN `ordered-map` (`@ordered %{ ... }`) — order is part of the
value's identity, unlike plain `map`. Wraps an ordered list of
`{key, value}` pairs rather than a plain Elixir map, so insertion
order survives structural `==` comparison.

  - [Dextrin.Rational](Dextrin.Rational.md): DXN `rational` (`int/uint`) — an exact ratio, stored exactly as
given and never silently reduced: `22/7` and `44/14` are distinct DXN
values (`DXN.md` calls it "exact ratio," not "exact reduced ratio"),
so reducing on parse would be a silent, opinionated transformation.
`reduce/1` is offered but never called implicitly.

  - [Dextrin.SortedSet](Dextrin.SortedSet.md): DXN `sorted-set` (`@sorted-set @{ ... }`). Wraps a list kept sorted
(and deduplicated) as a hard invariant at every construction site —
`new/1` is the *only* way to build one, precisely so that structural
`==` between two `Dextrin.SortedSet`s is valid set-equality rather
than something that happens to work only when both were built the
same way — every construction site (decode, encode, public API)
goes through `new/1`, so the sorted/deduplicated invariant can never
be bypassed.

  - [Dextrin.Struct](Dextrin.Struct.md): Opaque DXN `struct` value, produced when no schema is compiled for
its name (DXN.md §1.4: "a reader lacking the schema... returns an
opaque tagged value rather than failing").
  - [Dextrin.Symbol](Dextrin.Symbol.md): DXN `symbol` — a bare, unevaluated identifier reference. Wraps a
plain `String.t()`, never an Elixir atom: atoms are never garbage
collected on the BEAM, and a decoder fed adversarial or merely large
third-party input must not be able to exhaust the atom table by
decoding enough distinct symbols. A caller who trusts their input
and wants a real atom can always convert explicitly.

  - [Dextrin.Tuple](Dextrin.Tuple.md): DXN `tuple` (`{ ... }`) — ordered, heterogeneous, arbitrary length.
Wraps a list, not an Elixir tuple: DXN tuples have no fixed arity in
the type system the way Elixir's do, and `Dextrin.Array` already
claims the Elixir tuple for the one DXN type that actually is
fixed-size/indexed.

  - [Dextrin.Uri](Dextrin.Uri.md): DXN `uri` (`@uri "..."`, RFC 3986). Wraps the raw string exactly as
given, not parsed into `URI.t()` — `URI.parse/1` is lenient in ways
RFC 3986 isn't, and `URI.to_string/1` doesn't always reproduce the
original text byte-for-byte (e.g. component re-escaping), which would
risk breaking round-trip. Parsing is left to the caller, who can
always call `URI.parse/1` on `value` themselves.

  - [Dextrin.Uuid](Dextrin.Uuid.md): DXN `uuid` (`@uuid "..."`, RFC 4122). Wraps 16 raw bytes, not the
36-char text form — the text form exists only at the `.dxn`/`.dxnb`
encoding boundary (`parse/1`/`format/1` below convert between them);
the in-memory value stays the compact, comparison-cheap byte form.

- Text pipeline (.dxn)
  - [Dextrin.Text.Actions](Dextrin.Text.Actions.md): `Ichor.Actions` implementation for `Dextrin.Text.Grammar` — turns
the raw capture tree `Grammar.Native` produces while matching
`priv/grammar/dxn.aether` into actual `Dextrin.Value.t()` values.
`context` is a `Dextrin.Registry.t()` — text parsing only ever reads
it (to resolve struct schemas and custom tags), never mutates it.
  - [Dextrin.Text.Escapes](Dextrin.Text.Escapes.md): Shared escape-decoding for DXN's `string`/`char`/quoted-`keyword`
bodies (`DXN.md` §1.1's `escape` production) — one implementation so
none of the token handlers in `Dextrin.Text.Actions` duplicate it.

  - [Dextrin.Text.Formatter](Dextrin.Text.Formatter.md): Multi-line, indented `.dxn` rendering — `Dextrin.encode/2`'s
`pretty: true` opt, `mix dextrin.format`'s `--mode pretty`, and `mix
dextrin.decode`'s default output (a CLI decode is a human reading
the result, so `encode/2`'s own single-line default isn't the right
default there). `--mode condense` needs no separate implementation
at all: it's exactly what `Dextrin.encode/2`/`Dextrin.Text.Printer`
already produce with `pretty: false` (the default).
  - [Dextrin.Text.Grammar](Dextrin.Text.Grammar.md): Compiles `priv/grammar/dxn.aether` via Ichor's `Grammar.Native`
backend, not `Grammar.VM`'s interpreted backend — the grammar is
fixed at `dextrin`'s own build time (there's no scenario where a
caller supplies a different grammar at runtime), so there's no
reason to pay for bytecode interpretation when a compiled parser is
available for free.
  - [Dextrin.Text.Printer](Dextrin.Text.Printer.md): `.dxn` printer — the reverse of `Dextrin.Text.Actions`. Single-line,
minimal-whitespace output: a printer, not a formatter (no
line-wrapping or indentation policy is specified anywhere in
`DXN.md`) — `Dextrin.Text.Formatter.pretty/2` owns multi-line,
human-readable rendering, built on top of this module.

- Binary pipeline (.dxnb)
  - [Dextrin.Binary.Decoder](Dextrin.Binary.Decoder.md): `.dxnb` decoder — the mirror image of `Dextrin.Binary.Encoder`.
Envelope (`magic` `version` `cbor_item`) checked once at the top;
everything below that is a single recursive `decode_item/2`, keyed
on CBOR major type first, then on tag — matching `DXN.md` §2.2's
table row order the same way the encoder does.

  - [Dextrin.Binary.Encoder](Dextrin.Binary.Encoder.md): `.dxnb` encoder — a recursive walk over CBOR's major types plus a
fixed tag dispatch table, matching `DXN.md` §2.2's table row order.
  - [Dextrin.Binary.Tags](Dextrin.Binary.Tags.md): Pure constants — CBOR tag numbers and bit-layout tables, mirroring
`DXN.md` §2.3 verbatim. No behavior lives here.

- Schema (.dxns)
  - [Dextrin.Schema](Dextrin.Schema.md): Public entry point for `.dxns` schema compilation and validation.
  - [Dextrin.Schema.Compiled](Dextrin.Schema.Compiled.md): The result of compiling one `%schema{}` entry from a `.dxns`
document (`Dextrin.Schema.Compiler.compile/3`) — enough to both
validate a decoded value and convert it both directions between
`.dxnb`'s always-positional wire shape and a named field map.
  - [Dextrin.Schema.Compiler](Dextrin.Schema.Compiler.md): Compiles a parsed `.dxns` document (an ordinary decoded DXN value —
a `.dxns` file is valid `.dxn`, no new grammar; what makes it a
*schema* document is purely the shape of the value it parses to: a
map from name to type expression) into `Dextrin.Schema.Compiled`
entries.
  - [Dextrin.Schema.Field](Dextrin.Schema.Field.md): One compiled field spec inside a `Dextrin.Schema.Compiled` struct.
`required` comes from the `?`-suffixed key convention (a field key
ending in `?` is optional; no separate flag exists in `.dxns` itself
— reusing DXN's own identifier grammar rather than adding one);
`default`/`description` only ever come from the `%field{...}` escape
hatch, since optionality is already fully covered by the key suffix.

  - [Dextrin.Schema.FileResolver](Dextrin.Schema.FileResolver.md): An optional convenience `struct_resolver` resolving `Namespace/Name`
references to `.dxns` files on disk.
  - [Dextrin.Schema.Provider](Dextrin.Schema.Provider.md): Lets a struct's *own* library ship a DXN schema for it — field
names, types, required/optional, closed/forbidden, refinements, a
materializer — without that library ever taking a hard dependency on
`dextrin`. This is the `struct` (schema-backed) counterpart to what
`Dextrin.Registry.put_tag_encoder/4` already does for simple
scalar-wrapping `custom-tag` values: an extension point owned by the
*consuming* application, not forced onto every struct-defining
library it might want to use with `dextrin`.
  - [Dextrin.Schema.Std](Dextrin.Schema.Std.md): dextrin's standard library of named types (`priv/schema/std.dxns`)
— common refinements like `PositiveInteger` or `NonEmptyString`, so
a schema author doesn't redefine them by hand. Built the same way
any consumer's own named types would be — nothing about them is
special-cased in `Dextrin.Schema.Compiler`. Deliberately excludes
anything domain-specific (email, phone number, URL-shaped string):
what counts as a valid one is an application decision this library
shouldn't guess at.
Opt-in: pass `Std.registry/1`'s result as `compile/3`'s
`base_registry` to make these names available to your own document.
  - [Dextrin.Schema.TypeExpr](Dextrin.Schema.TypeExpr.md): Internal, compiled representation of a `.dxns` type expression — the
13-form vocabulary (`any`, `primitive`, `reference`, `list-of`,
`set-of`, `tuple-of`, `map-of`, `enum`, `one-of`, `all-of`,
`nilable`, `refine`, `struct`) `Dextrin.Schema.Compiler` turns a
parsed `.dxns` value into, and what `matches?/2` below checks a
decoded value against. This vocabulary is fixed, not extensible from
outside an actual library change — what schema *authors* extend
instead is composing these forms into a reusable named type (see
`Dextrin.Schema.Compiler`'s own moduledoc), which needs no new form
at all.

  - [Dextrin.Schema.Validated](Dextrin.Schema.Validated.md): Internal only — never appears in a value handed back to
`Dextrin.decode/2`/`decode_binary/2`'s caller.
  - [Dextrin.Schema.Validator](Dextrin.Schema.Validator.md): Checks a value's fields against a `Dextrin.Schema.Compiled` schema
(required/closed/forbidden/refine) — shared by decode (`materialize/4`,
which also produces a materialized result) and encode-time validation
(`validate_for_encode/3`/`validate_tree_for_encode/2`, which never
transforms `value`). Both directions reuse the exact same
`resolve_fields/3`/`TypeExpr.matches?/3` field-checking — encode's
own values are wrapped in the same internal `Dextrin.Schema.
Validated` marker decode already uses, via `wrap_and_check/2`, so
there's one type-checking implementation, not two that could drift.

- Unicode
  - [Dextrin.Unicode.RangeGenerator](Dextrin.Unicode.RangeGenerator.md): Pure text-processing core for turning a Unicode Character Database
`DerivedCoreProperties.txt` into the generated `IDENT_START`/
`IDENT_CONT`/`IDENTIFIER` block spliced into `priv/grammar/dxn.aether`.
No file or network I/O lives here — that's
`Mix.Tasks.Dextrin.Gen.Unicode`'s job — so this module can be tested
directly against small in-memory fixtures instead of the real,
~1MB UCD file.

## Mix Tasks

- Mix tasks
  - [mix dextrin.decode](Mix.Tasks.Dextrin.Decode.md):     $ mix dextrin.decode data.dxnb
    $ mix dextrin.decode data.dxnb --out data.dxn
  - [mix dextrin.encode](Mix.Tasks.Dextrin.Encode.md):     $ mix dextrin.encode data.dxn
    $ mix dextrin.encode data.dxn --out data.dxnb
    $ mix dextrin.encode data.dxn --share
  - [mix dextrin.format](Mix.Tasks.Dextrin.Format.md):     $ mix dextrin.format data.dxn --mode pretty
    $ mix dextrin.format data.dxn --mode condense
    $ mix dextrin.format data.dxn --mode pretty --in-place
  - [mix dextrin.gen.schema](Mix.Tasks.Dextrin.Gen.Schema.md):     $ mix dextrin.gen.schema MyApp.Point
    $ mix dextrin.gen.schema MyApp.Point --out point.dxns --name Point
  - [mix dextrin.gen.unicode](Mix.Tasks.Dextrin.Gen.Unicode.md): Fetches the latest Unicode Character Database `DerivedCoreProperties.txt`,
compares its version against the one last processed
(`priv/unicode/VERSION`), and — if newer — regenerates the
`IDENT_START`/`IDENT_CONT`/`IDENTIFIER` ranges spliced into
`priv/grammar/dxn.aether` (see `Dextrin.Unicode.RangeGenerator` for
the pure text-processing logic this task wraps with file/network I/O).
  - [mix dextrin.validate](Mix.Tasks.Dextrin.Validate.md): Decodes `PATH` (format sniffed from its extension, or forced with
`--format text|binary`), reporting success or a rendered
`Dextrin.Error` — non-zero exit on failure, so this is meant for CI
as much as interactive use.

