Codecs and schema evolution

This guide describes native Elixir values, Skir JSON and binary encoding, schema evolution, and serialization limits. Examples use User generated by skir-elixir-gen.

Data model and encoding

Dense JSON and binary use the compiler's field numbers, retain removed-slot positions, and omit trailing defaults. The binary API prefixes values with ASCII skir. Large 64-bit JSON integers use decimal strings outside JavaScript's exact integer range. Bytes use base64 in dense JSON and hex: in readable JSON. Finite float32 values are rounded when encoded to binary, not when encoded to JSON. Non-finite BEAM values are represented explicitly as atoms.

Nested struct defaults are emitted as literal struct-shaped maps, avoiding compile-order cycles between generated modules. Unavoidable hard-recursive defaults terminate in :skir_default. The encoder accepts that sentinel only in struct positions. Explicit default-only recursive chains may canonicalize back to the finite default; do not rely on their materialized depth surviving a round trip. Prefer optional links for ordinary application trees.

Keyed-array helpers build a map and raise ArgumentError on duplicate keys. Keys can follow nested fields and enum .kind. They do not automatically expand a hard-recursive :skir_default sentinel in a key path.

Evolution and unknown fields

Decoding discards unknown data by default, matching the conservative upstream policy. Preserve it explicitly only when the source is trusted:

old_reader = User.from_json!(newer_dense_json, unknown_fields: :preserve)
updated = %{old_reader | name: "Bob"}

# Retained fields survive same-format encoding without repeating the option.
forwarded = User.to_json!(updated)

# Dense unknown values have no type information for conversion to binary.
{:error, %Skir.Error{reason: :unknown_format}} = User.encode(updated)

# Explicitly authorize lossy conversion when that is appropriate.
binary_without_unknowns = User.encode!(updated, unknown_fields: :discard)

Unknown binary field values retain their exact original bytes. Unknown values are validated before being emitted. Removed fields/variants are discarded rather than treated as newly unknown fields. Readable unknown field names can be preserved, but readable JSON is not safe across field renames.

Constant-to-payload enum evolution is handled in both directions: reading the old constant with the new schema supplies a default payload; reading the new payload with the old constant schema ignores the payload. No atom is created from an incoming enum name or JSON field name.

Unknown preservation can otherwise allow an untrusted client to smuggle a future field through an older service. Do not enable it globally at a public API boundary.

Validation and limits

Serialization options:

[
  max_bytes: 4_194_304,
  max_depth: 64,
  max_collection_length: 100_000,
  max_nodes: 200_000,
  unknown_fields: :discard  # decode default; encode preserves already-retained data
]

JSON encoding additionally accepts format: :dense (default) or :readable. Binary APIs accept only format: :binary. Unknown options and invalid limits are errors.

Wire decoders bound input bytes, nesting, collection lengths, and aggregate parsed nodes, and reject trailing binary bytes. JSON nesting is pre-scanned before built-in JSON builds a tree. UTF-8, base64, numbers and payload shapes are checked. The independent agent source review and deterministic malformed-input checks cover these defenses; they are not third-party security certification or a substitute for deployment HTTP body/time limits. Runtime 0.2.1 bounds RPC method-number conversion and default HTTP response allocation, including chunked and error responses; see the RPC guide. Native to_json/3 receives an already-allocated application tree; its byte limits on terms are not a wire-size guarantee for the whole returned tree. Apply transport limits before parsing untrusted requests.

This runtime intentionally rejects some malformed/out-of-range values that upstream implementations may coerce or clamp. It does not claim identical error permissiveness on invalid inputs. For very large valid timestamps beyond Elixir's calendar range, readable output contains unix_millis without formatted.