Modules
use ALLM.Pipeline — the framework's notion of "a pipeline".
The shared wrapper over ALLM.Pipeline.Artifacts — stores and retrieves
pipeline artifacts (HTML, JSON, extracted text) through the configured
adapter, and returns a URL reference for the PostgreSQL step_log.
Storage behaviour for pipeline artifacts — the heavy bodies a step_logs row
deliberately does not carry (scraped HTML, extracted PDF text, decision JSON,
the per-step LLM-call envelope).
DynamoDB client for artifact storage.
Local-disk ALLM.Pipeline.Artifacts adapter — the one that lets a fresh
clone run pipelines with zero cloud infrastructure.
In-VM ALLM.Pipeline.Artifacts adapter — artifacts live in an Agent and
die with it.
S3 ALLM.Pipeline.Artifacts adapter — the large-tier backend for artifacts
that do not fit a DynamoDB item.
A size-routing ALLM.Pipeline.Artifacts adapter — small artifacts go to one
backend, large ones to another. It is what makes the tier list an adapter
choice rather than the hard-coded S3 residue ALLM.Pipeline.ArtifactStore
used to carry (architecture §3.6, §2.7).
Resolves the host-supplied collaborators the package cannot name directly.
Pipeline execution context passed to each step — and, since Phase 4, to every
ALLM.Pipeline escape-hatch body and FanOut.reduce/5 fold function as well.
The construct macros behind use ALLM.Pipeline, plus the compile-time
validators.
One item's outcome from a fan_out, and the carrier for per-item lineage.
One declared resource — a handle acquired once per run and released before
the terminal write.
The interpreter the generated run/1 calls.
One compiled stage of an ALLM.Pipeline declaration.
The single implementation of "make an arbitrary term safe for a jsonb column".
Base utilities for executing pipeline steps.
The canonical home for the fan-out safety rule, plus the one helper
every Task.async_stream site shares.
The seam through which the package calls a host's LLM engine.
Per-step collector for full LLM-call inputs/outputs.
A macro for the LLM-calling half of an ALLM.Pipeline.Step.
The run lifecycle guard, as ONE implementation with two consumers.
Serialization guard for pipeline runs — the abstraction behind "don't let two runs of the same pipeline overlap on the shared database".
Postgres session-advisory-lock implementation of ALLM.Pipeline.Lock.
No-op ALLM.Pipeline.Lock — the current default.
Record and query normalized pipeline metrics (the found → mapped → processed funnel).
Emission point: a pipeline calls record/3 at the SAME place it calls
PipelineRun.complete/2 (the sole completer for umbrella/borrowed runs).
One normalized metrics row for a pipeline run: the found → mapped → processed funnel (plus skipped/failed) for a single entity type. See the "Metric semantics" section of steering/PIPELINE_METRICS_DASHBOARD.md.
Represents a single execution of a pipeline.
The package-owned, read-only query facade for host UIs and host contexts.
The host's one declaration of how it wires the framework, and of the domain knowledge the framework must not carry itself.
A macro for defining Input/Output schemas with reduced boilerplate.
Derives an OpenAI strict-mode JSON schema from an ALLM.Pipeline.Schema
declaration.
Behavior for pipeline steps with strongly-typed inputs and outputs.
Persists step execution logs to PostgreSQL for observability and lineage tracking.
Persistence behaviour for runs and steps — the observability record a pipeline leaves behind.
PostgreSQL implementation of ALLM.Pipeline.Store, and the adapter that
ships the package's run/step schemas.
The [:allm_pipeline, …] telemetry event contract, plus thin emit_*
helpers wrapping :telemetry.execute/3.
Batteries-included text handling for pipeline persistence: scrubbing text so Postgres will accept it, whitespace normalization, and head+tail truncation for oversized artifact bodies.
Mix Tasks
Emits the installed ALLM.Pipeline.Registry's per-pipeline metadata as a
machine-readable list of names.
Reports ALLM.Pipeline.Schema's narrow nilability rule across every compiled
schema module in the current project.