ALLM.Pipeline.LLMStep (allm_pipeline v0.1.0)

Copy Markdown View Source

A macro for the LLM-calling half of an ALLM.Pipeline.Step.

An LLM step used to carry four parallel artifacts: an Input struct, an Output struct, a hand-written strict-mode JSON schema, and — where the Output had an enum — a per-module string→atom coercion map. Phase 2 made the Output declaration authoritative for the struct and its types; Phase 3.1 made it authoritative for the JSON schema. This macro makes it authoritative for the parse path too, so a ported step carries one declaration and a prompt.

defmodule MyStep do
  use ALLM.Pipeline.LLMStep,
    type: :transform_ordinance,
    input: __MODULE__.Input,
    output: __MODULE__.Output,
    engine: :nano,
    schema_name: "ordinance"

  @impl true
  def prompt(%Input{} = input), do: "…"
end

What it generates

FunctionContract
step_type/0, input_schema/0, output_schema/0the ALLM.Pipeline.Step callbacks, from the use options
json_schema/0the Output's derived strict-mode schema — output.__allm_schema__(:json_schema)
call_llm/1prompt/1ALLM.Pipeline.LLM.impl().generate_structured/4; {:ok, parsed, tokens} or {:error, {:llm_error, reason}}
coerce/2parsed payload + token count → the Output struct
post_process/2identity; the ordinary hook, overridable
execute/2a thin composition of the three above, overridable

It also injects @behaviour ALLM.Pipeline.Step. That is not decoration: a consumer repo's Step-schema census test derives the Step population two independent ways — by the attribute and by the exported callbacks — and asserts the sets are equal, so generating the callbacks without the attribute fails that test by name.

Required of the using module: prompt/1, taking the Input struct and returning the prompt string or a message list. Its absence is a compile error naming the module.

Checked at compile time, all naming the offending module: input: names a compiled struct module; output: names a module that answers __allm_schema__(:json_schema) (i.e. declared json_schema: true); that Output declares tokens_used, if at all, as wire: false; and prompt/1 exists. The input:/output: probes both go through Code.ensure_compiled/1 so a sibling-file module compiled in the same batch resolves.

⚠️ post_process/2 is not a Step callback — override it with a plain def, without @impl. @impl true on it emits "no behaviour specifies such callback", which --warnings-as-errors turns into a build failure. The overridable that is a callback, and therefore does take @impl true, is execute/2.

Why the parse path is separate PUBLIC functions

execute/2 is overridable because real steps need control flow — the canonical case is a transformer step that makes a conditional second call when the model claims an appointment but omits its details. A wholesale override replaces the generated body, so if the coercion lived inside execute/2 an overriding step would have to hand-write it again and the macro would buy that step nothing.

So an overriding execute/2 calls coerce/2 itself, and gets the wire-name mapping, the enum coercion and the token plumbing for free. llm_step_test.exs pins that an override calling coerce/2 produces a struct identical to the generated path's.

What coerce/2 does, field by field

It walks the Output's declared fields (__allm_schema__(:fields)) and consults :wire, :values and :generated_types:

  • wire: false — never read from the payload. The field is populated by the harness, not the model (bill_number from the Input, tokens_used from the response envelope), so the derived schema omits it and so does this.
  • wire: "summary" — read from parsed["summary"] into the field named ai_summary. The wire contract and the struct's field names are allowed to differ, and on every real transformer they do.
  • an absent or null payload value — the field is left out of the struct entirely, so its declared default: applies. This is what the hand-rolled response["x"] || false / || [] idioms did at every retired call site. A null element inside an array is dropped from the list for the same reason, for every list type (see the table below).
  • atom() with values: — the string is matched against the declared vocabulary. Never String.to_atom/1 or String.to_existing_atom/1 on model output: the vocabulary is the allowlist. An unrecognized string becomes :other iff :other is a declared member, and is otherwise a coercion failure that fails the step.
  • String.t() with values: — emitted as a JSON enum, stored unchanged. projects.scale is a string column and coercing it to an atom would change what the loader writes.
  • Date.t() — parsed from ISO-8601. An unparseable value degrades to nil rather than failing the step, matching the parse_date/1 helpers this replaces; the field is nullable in the derived schema either way.
  • tokens_used — populated from the response envelope iff the Output declares the field. MeetingImportanceScorer.Output does not, and that must not raise. An Output that declares it must declare it wire: false; the macro refuses to compile otherwise, because the derivation would otherwise ask the model to invent a count coerce/2 immediately overwrites.
  • a scalar where the type says list — a coercion failure ({:not_a_list, raw}), again for every list type. Strict mode declares "type": "array", so this is a non-compliant payload, and coerce/2 is the boundary that says so.

Which list types coerce/2 inspects — all of them

The two rules above are true of every declared list type. coercion/1 builds a {:list, kind} for any element type; the element kind decides only what happens per element. :atom and :date elements are converted, everything else — [String.t()], a nested schema module, a nested list — is the identity (coerce_scalar(:passthrough, raw, _)). The two list-level rules are applied by read/3 and map_ok/3 before the element kind is consulted, so they do not vary with it. Measured 2026-08-19, not inferred:

Declared typecoercion/1["a", nil, "b"]a bare "a"
[atom()] with values:{:list, :atom}[:a, :b]nil droppedfails: {:not_a_list, "a"}
[Date.t()]{:list, :date}[~D[2026-01-01]]nil droppedfails: {:not_a_list, "a"}
[String.t()]{:list, :passthrough}["a", "b"]nil droppedfails: {:not_a_list, "a"}

This uniformity is deliberate and was widened on 2026-08-19 (a user decision, prompted by the 3.3 review's F3). [String.t()] used to collapse to a bare :passthrough, which made both rules unreachable for the one list type every ported step actually declares — the doc said "an array" and meant "[atom()] or [Date.t()]". The tradeoff accepted with it: a malformed payload on a [String.t()] field that used to be absorbed silently now fails the step with {:error, {:coerce, [{field, {:not_a_list, raw}}]}}.

Consequence when porting a step. A hand-rolled filter_strings/1-shaped defense on a list field is no longer load-bearing against null elements or a bare scalar — coerce/2 is the boundary for those, whatever the element type. It may still be doing other work (OrdinanceTransformer's drops any non-string element a json_schema:-hatched field could still admit, and its parse_provisions/1 / parse_areas/1 rebuild nested structs), and post_process/2 is public, so a caller can hand it a struct coerce/2 never touched. Keep the clauses; do not delete one merely because the coercion path can no longer reach it.

The struct is built with struct/2, not struct!/2: a required: true field that is also wire: false (bill_number) is legitimately unset at this point and is filled by post_process/2 or by an overriding execute/2.

The boundary: nested schema modules are NOT rebuilt

A field typed by a nested ALLM.Pipeline.Schema module keeps the model's raw string-keyed map. coerce/2 resolves no aliases — it runs at runtime, where the Macro.Env that ALLM.Pipeline.Schema.JsonSchema expands short aliases against no longer exists — so it cannot tell KeyProvision inside a parent from a top-level module of that name. Rebuilding the nested struct is therefore the step's work, in post_process/2, which is where every transformer this replaces already did it (parse_provisions/1, parse_appointment_details/1).

Declare the STRUCT type[KeyProvision.t()] — and pin the wire shape with the per-field json_schema: hatch, so the generated @type t describes what post_process/2 returns rather than what the wire carries; the two overlap at [], so declaring the wire type instead is invisible to dialyzer. (This paragraph read "declare such a field [String.t()]" until 2026-08-19; the ordinance port measured the hatch byte-identical on the wire and compile-enforced — dropping it is an ArgumentError, since the nested struct does not declare json_schema: true.) Such a field still classifies {:list, :passthrough}, so the LIST-level rules above apply to it and its ELEMENTS still arrive as the model's raw maps.

The engine name is the host's vocabulary

engine: :nano is resolved through ALLM.Pipeline.LLM.impl().resolve_engine/1 at call time. The package neither knows nor validates the names — see ALLM.Pipeline.LLM.

Summary

Types

A validated use declaration.

Functions

Generate the Step callbacks, the engine call and the parse path.

Types

declaration()

@type declaration() :: %{
  type: atom(),
  input: module(),
  output: module(),
  engine: atom(),
  schema_name: String.t()
}

A validated use declaration.

:input and :output are module names resolved in the using module's context, so __MODULE__.Output arrives here already expanded.

Functions

__using__(opts)

(macro)

Generate the Step callbacks, the engine call and the parse path.

Options — all five are required: :type (the step_type/0 atom), :input and :output (schema modules), :engine (a host engine name) and :schema_name (the strict-mode schema's name on the wire).