A macro for defining Input/Output schemas with reduced boilerplate.

Usage

defmodule MyApp.MyStep.Input do
  use ALLM.Pipeline.Schema

  schema do
    field :name, String.t(), required: true
    field :count, integer(), default: 0
    field :bulk, [map()], log: false
    field :api_key, String.t(), redact: true
  end
end

Options

• json: true - Adds @derive Jason.Encoder to make the struct JSON-encodable
• json_schema: true - Derives __allm_schema__(:json_schema) — see "The derived JSON schema is opt-in" below

An unknown use option raises ArgumentError, so json_schmea: true fails loudly rather than silently leaving the schema underived.

Field Options

An unknown field option raises ArgumentError at the using module's compile time, naming the option and the field. So does a non-boolean value on any of the five boolean options (everything except default: and the four LLM-facing options): redact: "true" is a compile error rather than a silently-unredacted field.

The LLM-facing options

values:, description:, wire: and json_schema: exist to make the OpenAI strict-mode JSON schema a derived artifact of the declaration rather than a second hand-maintained description of the same shape. They generate nothing on their own; ALLM.Pipeline.Schema.JsonSchema reads them.

values: takes a compile-time expression, not only a literal list — field/3 unquotes its options, so values: Schemas.Ordinance.fiscal_impacts() is evaluated at the using module's compile time and validated as the resolved list. That is deliberate: the real vocabularies in this project are accessor calls with a single owner, and requiring a literal would prescribe a copy of a list that already has one. The list must be non-empty and homogeneous — all atoms or all binaries — and may not contain nil, because a null member is derived from the field's nilability, never declared (is_atom(nil) is true, so a bare list-of-atoms check would wave it through).

Strings are the common case: projects.scale is a string column, so its field is String.t() with a string values: and no atom coercion. Atom coercion is a property of the declared type (atom()), not of values:.

wire: false marks a field the harness populates rather than the model — a tokens_used read off the response envelope, or a key copied from the Input. Demanding it in the schema's required would ask the model to invent it.

The derived JSON schema is opt-in

__allm_schema__(:json_schema) exists only on a module declaring json_schema: true; on any other schema the key raises FunctionClauseError like any unknown key. The opt-in is not ceremony: an unmappable type is a compile error (an open map() / term() / bare list() has no closed strict-mode rendering), and most schemas in this tree legitimately carry such fields because they never cross a wire. Deriving for every schema would turn each of those into a build failure.

The narrow nilability rule

A field's generated @type t entry gains | nil iff it has neither required: true nor a non-nil default:, its declared type does not already end in | nil, and it carries no explicit nilable: option. nilable: true forces the tail — even onto a required: field — and nilable: false forbids it. A field with a default always holds that default, and a required field is non-nil by construction, so the rule adds | nil exactly where a value can genuinely be absent at runtime.

Two readings the rule depends on, both deliberate:

• default: nil is not a default (the producers test != nil), so field(:engine, term(), default: nil) gains | nil.
• default: false IS a default (false != nil), so it does not.

The rule is applied here in process_fields/1 and rewrites no field/3 source line. Consequences: a hand-written | nil is detected and left alone rather than doubled; __allm_schema__(:types) keeps reporting the declared AST, while :generated_types reports what was spliced; and every field declared after this inherits the rule without an edit.

mix allm_pipeline.nilability --report prints the current state per module.

The three states of log:

ALLM.Pipeline.StepLog keeps heavy bodies out of input_data / output_data with two layers: the per-field flags below, and a retained package-level fallback list of generic field names (:raw_html, :html, :content, :engine) that applies to every struct it serializes — including DSL structs, and including plain defstruct / Ecto structs it recurses into, which have no flags at all.

So log: is deliberately three-state rather than boolean:

• log: false — never in the row.
• log: true — in the row even if the field's name is on the fallback list. Without this state, "this field is named :content but must be logged" is unsayable, which would reproduce the global-by-name wart for those four names.
• unset — the fallback decides.

artifact: true generates nothing

It declares intent and feeds __allm_schema__(:artifact), and it implies log: false. It does not generate an artifact_content/1: every Step module implements that callback by hand, so a generated one would have no consumer. Combining it with log: true is a compile-time error — the two express opposite intents and the serializer's drop set would be contradictory.

redact: true applies at serialization, not at construction

The value is replaced by the literal string "[REDACTED]" when ALLM.Pipeline.StepLog serializes the struct — construction-time scrubbing would destroy the value the field exists to carry. Four paths were therefore not covered; subphase 2.3 closed the second in code and three remain uncovered, none of which the flag can reach:

1. artifact_content/1 — an opaque binary the Step builds itself. The rule that replaces coverage is documentation: a redact: true field must not be included in artifact_content/1.
2. ALLM.Pipeline.Executor's validation error messages — closed in subphase 2.3. They used to inspect/1 the rejected term into step_logs.error and the logs; Executor's render_shape/1 now renders the term's type and key NAMES only, never its values. Listed rather than deleted because it is the one of the four that a code fix could reach, and the reason it could is that the Executor owns the whole message.
3. ALLM.Pipeline.Executor.log_summary/4, which writes a caller-supplied map straight to output_data without passing through the serializer at all.
4. Inspect and exception messages. DSL structs derive no Inspect exclusion, so a redacted value renders in full under inspect/1 — and new/1 routes through struct!/2, whose KeyError on an unknown key renders the partially-built struct, secret included. Use cast/1 rather than new/1 on any input that mixes a redacted field with untrusted keys.

Generated Functions

• new/0 — a struct with defaults (only when there are no required fields)
• new/1 — a struct from a keyword list, or from a map with atom or string keys. Raises on an unknown key (struct!/2 semantics) and on a field supplied twice (both key forms in a map, or repeated in a keyword list)

• cast/1 — {:ok, t()} | {:error, [issue()]}; see below

• __allm_schema__/1 — introspection; see below

cast/1 does not interpret the declared type

cast/1 checks the shape of its input and nothing else:

• the input is a map (atom or string keys), a keyword list, or an existing %__MODULE__{} — otherwise {:error, [{:__input__, :not_castable}]}
• every key resolves to a declared field — otherwise {key, :unknown_field} (unknown keys are an error, never silently dropped)
• no field is supplied twice — under both its atom and its string key in a map, or repeated in a keyword list — otherwise {field, :duplicate_key}
• every required: true field is present and non-nil — otherwise {field, :missing}
• a struct of a different module → {:error, [{:__struct__, :wrong_struct}]}

It takes a term(), not map() | keyword() | t(): it is called from ALLM.Pipeline.Executor.validate_input/2 with whatever a caller handed run_step/5, which runs BEFORE the Executor's try/rescue — so it never raises, and the :not_castable arm is the reachable answer for anything else.

It performs no runtime checking of the declared type. The declared types are arbitrary quoted AST ([map()], String.t(), term(), module-qualified aliases); interpreting them at runtime means writing a type checker, dialyzer already covers them statically, and a misinterpretation would reject live production input at the top of ALLM.Pipeline.Executor.run_step/5.

cast/1 returns all issues, not the first one.

__allm_schema__/1, and why it is not __schema__/1

Every Ecto.Schema module also exports __schema__/1, and ALLM.Pipeline.StepLog recurses into live Ecto structs. A function_exported?(mod, :__schema__, 1) predicate would classify every Ecto struct as DSL-owned; __schema__(:fields) then succeeds with a colliding shape while __schema__(:dropped) raises FunctionClauseError — on the un-rescued step-log write path. The distinct name is what makes the predicate correct. Do not "simplify" it back.

An unknown key raises FunctionClauseError, so a typo fails loudly.

:declared_logged is deliberately not called :logged. It reports what the field flags say and ignores the retained fallback list, so for a struct with an unflagged field :content, String.t() it lists :content even though :content never reaches output_data. The persisted set is computable only with the fallback in hand, which is the serializer's job, not the schema's.

:types vs :generated_types diverge for exactly the fields the nilability rule fires on. They are separate keys because that rule is applied by this macro and rewrites no source: the declared AST is byte-identical before and after, so only the generated AST can show the change. Use :types to ask "what did the author write?" and :generated_types to ask "what does dialyzer see?".

Not implemented here

The coercion half of values: and wire: — reading a model's parsed payload back into this struct — belongs to ALLM.Pipeline.LLMStep (extraction plan Phase 3.2), which reads __allm_schema__(:values) and __allm_schema__(:wire). This module records the declaration; it never parses a payload.