Changelog

Copy Markdown View Source

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

[0.1.0] - 2026-07-30

Added

  • .dxn text: full grammar support for all 30 types DXN.md §1.3 defines, via a hand-authored Aether grammar (priv/grammar/dxn.aether) compiled at build time through Ichor's Grammar.Native backend.

  • .dxnb binary: a hand-rolled CBOR codec (Dextrin.Binary.Encoder/ Decoder) covering the full type mapping, the private tag block (200-214), the duration bitmask and regex flags byte, and both value-sharing extensions (string-only, tag 256/25; general arbitrary- value sharing, tags 28/29 — decode-only required, encode opt-in via share: true, gated by a size-aware threshold rather than a fixed rule).

  • .dxns schema documents: Dextrin.Schema.compile/3 compiles a decoded .dxns document (itself plain .dxn data, no new grammar) into a Dextrin.Registry — struct schemas with required/optional (?-suffixed keys)/closed/forbidden fields and refine constraints, plus reusable named types composed from the fixed 13-form type_expr vocabulary. Enforcement is automatic, fail-fast, and symmetric: Dextrin.decode/2/decode_binary/2 check every registered struct name unconditionally on the way in; Dextrin.encode/2/ encode_binary/2 do the same automatic whole-tree check on the way out (validate: false opts out), plus an opt-in schema: check for a nameless top-level value.

  • Dextrin.Registry: the shared extension point for both struct (schema-driven, put_struct_materializer/3/put_struct_module/3) and custom-tag (put_tag/3/put_tag_encoder/4), plus a lazy put_resolver/2 hook for on-demand schema loading.

  • Dextrin.Schema.Std: a small standard library of common named types (PositiveInteger, NonEmptyString, Percentage, ...), opt-in via Std.registry/1.

  • Dextrin.Schema.FileResolver: one reasonable, swappable convention resolving Namespace/Name references to <path>/Namespace.dxns files on disk.

  • Value types: Dextrin.Symbol, Dextrin.Keyword (never Elixir atoms — decoding untrusted data can't exhaust the atom table), Dextrin.Tuple, Dextrin.Array, Dextrin.OrderedMap, Dextrin.SortedSet, Dextrin.Struct, Dextrin.Duration, Dextrin.Rational, Dextrin.Uuid, Dextrin.Uri, Dextrin.Bytes, Dextrin.Char, Dextrin.CustomTag — small wrapper structs only where Elixir has nothing native that fits without losing information.

  • Dextrin.Text.Formatter: multi-line, indented .dxn rendering (pretty/2), built on top of Dextrin.Text.Printer's single-line default.

  • mix dextrin.* tasks: validate, encode, decode, format, gen.schema (scaffold a .dxns file from an existing Elixir struct's field list), and gen.unicode (regenerate the grammar's Unicode XID_Start/XID_Continue identifier ranges from the latest Unicode Character Database — a deliberate, reviewed action, never run at build time).

  • Conformance fixtures covering DXN.md §3's full worked example, per-type round-trip tests, cross-format equivalence tests, grammar hazard regression tests, and a fuzz/malformed-input pass over the binary decoder.

  • Dextrin.encode/2 gained pretty: and indent: opts — pretty: true produces multi-line, indented output (indent: sets spaces per level, default 2) instead of the single-line/compact default. Either way, schema validation runs the same; formatting and validation are independent concerns.

  • Dextrin.Schema.Provider, a behaviour letting a struct's own library ship a DXN schema for it without that library ever depending on dextrin — the schema, the target struct module, and an optional materializer are declared on a small, separately-compiled companion module (meant to be guarded behind an optional: true dependency), and Dextrin.Schema.register_provider/2 compiles and wires it into a Dextrin.Registry in one call from the consuming application.

  • Property-based tests (stream_data/ExUnitProperties, already a declared dependency but previously unused): a shared Dextrin.Generators module, round-trip properties covering encode/2/decode/2, encode_binary/2/decode_binary/2, share: true, and pretty: true against randomly generated values, a Dextrin.Binary.FuzzTest asserting decode_binary/2 never raises or hangs on arbitrary or bit-flipped input, and a per-type property suite (Dextrin.RoundTripByTypeTest) covering every DXN type individually rather than mixed into one nested-structure generator. These found several of the bugs listed below on their very first run.

  • Dextrin.encode/2/encode_binary/2 accept a bare Elixir atom anywhere a keyword value is expected — as a map key (%{x: 1}, matching the shorthand's own colon syntax) or standalone (Dextrin.encode(:ok)) — exactly as if it were Dextrin.Keyword.new(Atom.to_string(atom)). DXN.md §1.3's own type table documents keyword's Elixir counterpart as "Elixir atom". nil/true/false are unaffected — those stay their own literals.

  • Dextrin.decode/2/decode_binary/2 gained a trusted: opt (default true) mirroring the above on the way in: keyword decodes as a real Elixir atom by default, the same natural type encode/2 now accepts. Pass trusted: false — per call, or once on a reused registry via the new Dextrin.Registry.put_trusted/2 — for any source you don't fully control; only then does keyword fall back to Dextrin.Keyword.t(), so a String.to_atom/1 call is never reachable from attacker-controlled text. symbol is unaffected either way — it stays Dextrin.Symbol.t() regardless of trusted, mirroring encode/2's own choice to treat a bare atom as a stand-in for keyword only, never symbol.

Changed

  • Breaking: Dextrin.Registry.new/0's trusted field — and so decode/2/decode_binary/2's effective default — is true, not false. Any code relying on the previous behavior (keyword decoding as Dextrin.Keyword.t() with no opt given at all) needs trusted: false added explicitly. This is deliberate: keyword's wrapper only ever existed to keep untrusted input from reaching String.to_atom/1 unbounded; assuming a trusted source (your own config, your own application's data) by default matches how the type is documented (DXN.md §1.3) and how most callers actually use this library, at the cost of needing an explicit opt-out for genuinely untrusted/network input.

  • Breaking: Dextrin.Text.Printer's (and so Dextrin.encode/2's default, non-pretty: output) is now maximally compact: no space after a map/struct entry's :, no space around =>, and entries/ positional-struct fields are comma-joined with no trailing space (%{x:1,y:2}, %Point{x:1,y:2}, %{1=>"one"}) rather than the previous %{x: 1, y: 2}-style spacing. DXN's own grammar treats whitespace and commas as insignificant everywhere, so this loses no information and every value still round-trips identically through decode/2; only the default rendering got smaller. List/tuple/ set/array element separators and @tag value's space are unchanged (the former have nothing to gain — comma and space are both a single byte — and the latter's space can't be dropped in general for a custom tag whose value could start with an identifier character).

  • Breaking: Dextrin.Text.Formatter.pretty/2 now returns {:ok, String.t()} | {:error, Dextrin.Error.t()}, matching Dextrin.Text.Printer.print/2's contract, instead of a bare String.t() that raised ArgumentError on an unencodable value. Also gained the same indent: opt Dextrin.encode/2's new pretty:/indent: opts use internally (default 2 spaces per nesting level, previously a fixed, non-configurable 2).

  • Adopted ichor's new mix ichor.gen/ichor_runtime split: the .dxn lexer/parser is now generated ahead of time into lib/dextrin/text/grammar/native.ex (checked in, regenerated via mix ichor.gen whenever priv/grammar/dxn.aether changes) instead of being produced by use Ichor at dextrin's own compile time. Dextrin.Text.Grammar is now a thin, hand-documented wrapper around the generated Grammar.Native. This lets mix.exs depend on the small ichor_runtime package (the only thing the generated code actually calls) as an ordinary runtime dependency, while ichor proper (the Aether front-end, format importers, Grammar.Analysis, both codegen backends) moves to only: :dev, runtime: false — the bulk of Ichor no longer ships in a dextrin release. Both are now published to Hex separately (ichor ~> 0.2.1, ichor_runtime ~> 0.1.0) and referenced as ordinary Hex dependencies — no git:/ sparse:/override: needed, since ichor's own mix.exs now depends on ichor_runtime the same way. Migrating surfaced a genuine gap in that split, fixed upstream: the raw-capture-node re-evaluation entry point needed by Dextrin.Text.Actions at actual decode time (for @ordered %{...}) had stayed on the top-level, dev-only Ichor module instead of moving to ichor_runtime; it's now Ichor.Actions.evaluate_node/3.

  • Added credo, dialyxir, sobelow, and excoveralls as dev/test tooling, plus a mix precommit alias (format, compile --warnings-as-errors, credo --strict, sobelow, test, dialyzer) run before every commit. Fixed everything it surfaced: canonical module layout (moduledoc/use/alias/...) across the mix tasks and text pipeline modules; Dextrin.decode/2's spec under-declared its own return type (the underlying grammar engine can report a list of errors, not just one) — widened to match, which also resolved three "dead code" warnings in the mix tasks' own error handling; removed a dead Duration.microsecond || {0, 0} fallback (that field is never nil).

  • Every hand-rolled "walk a collection, thread an accumulator, halt on the first non-{:ok, _} step result" reduce across Dextrin.Schema.Compiler, Dextrin.Schema.Validator, Dextrin.Binary.Encoder, Dextrin.Text.Printer, and Dextrin.Text.Actions now uses Ichor.Toolkit.Result.reduce_ok/3/ map_ok/3 instead of a hand-rolled Enum.reduce_while/3.

  • Documentation reworked throughout: every module's docs are now self-contained (no references to an external design document), and the normative DXN.md format specification moved under guides/dxn/DXN.md, alongside a full set of tutorials, examples, and cheatsheets for both this library and the DXN format itself. DXN.md itself gained a new normative §4, "Schema documents (.dxns)" — the type_expr/refine-constraint vocabulary, %schema{}/%field{} shape, named types, cross-file references, and enforcement semantics, written implementation-independent (this library's own choices stay documented separately, in Dextrin.Schema's own module docs) — closing a gap where .dxns was documented everywhere except the one place meant to be the authoritative, cross-implementation reference for it.

Fixed

  • Struct literal parsing (%Point{x: 1, y: 2}) and .dxns schema compilation both depended, internally, on keyword/map-key text always decoding as Dextrin.Keyword/Dextrin.Symbol — exposed as a real regression the moment trusted: true became the default, not merely a test-assertion mismatch: struct literals failed to parse at all, and any schema whose own .dxns source was decoded trusted failed to compile. Fixed at the boundaries that actually needed it: Dextrin.Text.Actions's struct-field-name resolution now accepts a bare atom the same way it already accepted Dextrin.Keyword /Dextrin.Symbol; Dextrin.Schema.Compiler.compile/3 normalizes a decoded .dxns document's atoms back to Dextrin.Keyword once, up front (mirroring Dextrin.Schema.Validated.strip/1's exact recursive-walk shape), so its own .dxns-vocabulary pattern matching needs no changes at every site; a schema's enum literals and a :keyword-typed field's actual data can now be either shape independently, compared correctly either way instead of only ever matching the untrusted shape. default: nil/true/false field values are deliberately not touched by that normalization (only .dxns syntax positions are) — and x: :nil as a schema field's type (the "nil" primitive) has its own dedicated fix, since String.to_atom("nil") and the bare literal nil are, once decoded, the exact same unrecoverable Elixir value.

  • Dextrin.Binary.Decoder's handling of the time tag (210) had no fallback clause — a malformed payload that wasn't an integer crashed with FunctionClauseError instead of a clean {:error, _}, the one tag missed in the earlier decoder-hardening pass. Found by the property-based fuzz test firing again after the trusted: default flip added more generated shapes to exercise it with.

  • Dextrin.Text.Printer/Dextrin.Text.Formatter no longer produce unparseable text for a map entry or struct field whose Dextrin.Keyword/field name isn't a bare identifier (empty, containing spaces, ...) — map_entry's colon-shorthand (DXN.md §1.2) only exists for an identifier; both printers now fall back to the arrow form (:"name" => value) for anything else, instead of emitting invalid syntax like %{"":0} or a bare : 0 that failed to parse back at all. Dextrin.Text.Printer.bare_identifier?/1 is now public so both printers share one rule.

  • Dextrin.Binary.Decoder no longer crashes (raises) on a malformed .dxnb document that places a tuple/array/set/sorted-set tag around a non-array payload, a bignum/uuid tag around a non-bytes payload, a uri/symbol/keyword tag around non-text, a char tag around a non-codepoint or out-of-range value, a date tag around a non-integer, or a timestamp/datetime epoch outside the representable range — every one of these now returns a clean {:error, %Dextrin.Error{}} instead of an unhandled FunctionClauseError/ArgumentError/CaseClauseError. Found by the new property-based fuzz test, not a hand-picked case.

  • Dextrin.Text.Printer.print/2 no longer crashes on a value with no DXN representation at all (a PID, a port, a reference, a function, a raw Elixir tuple that isn't Dextrin.Tuple, ...) — a clean {:error, %Dextrin.Error{}} now, matching encode/2's own typespec, instead of an unhandled FunctionClauseError.

  • @ordered %{} (an empty ordered map) failed to parse at all ("@ordered requires a map literal argument"), even though %{} on its own decodes fine — the action handler required its raw capture tree to have a :map_entry key present, which a zero-entry map simply doesn't have (the same reason the ordinary, non-@ordered map_lit handler already defaults it to []). Found by the new per-type property suite.

  • Dextrin.Binary.Decoder now accepts DXN.md §2.4's CBOR string -reference sharing extension (tags 256/25) — previously an otherwise-well-formed .dxnb document using it (to compactly represent repeated symbol/keyword text or repeated struct type names) failed to decode at all ("unrecognized CBOR tag 25"), which violated §2.4's "a conforming decoder MUST accept it" requirement. dextrin's own encoder still never produces tags 256/25 (that remains out of scope, same as before) — this is decode-side acceptance only, same stance as §2.5's tag 28/29 support.

  • Dextrin.encode/2/encode_binary/2 can now encode a registered application struct (Dextrin.Registry.put_struct_module/3) directly — previously, only a hand-built Dextrin.Struct could actually be serialized; a real struct registered via put_struct_module/3 was recognized by automatic encode-time validation but had no path to actually being written out, forcing callers to reconstruct a Dextrin.Struct from the real struct's own fields by hand before encoding. Dextrin.Text.Printer/Dextrin.Binary.Encoder now rebuild the equivalent Dextrin.Struct automatically, in the compiled schema's own canonical field order.