# allm_pipeline v0.1.0 - Table of Contents

> A step-based LLM pipeline framework: typed steps with persistent step logs, artifact lineage, run lifecycle ownership, and a declarative pipeline DSL.

## Pages

- [ALLM.Pipeline](readme.md)
- [Wiring a host](host_wiring.md)
- [CHANGELOG](changelog.md)

## Modules

- [ALLM.Pipeline](ALLM.Pipeline.md): `use ALLM.Pipeline` — the framework's notion of "a pipeline".
- [ALLM.Pipeline.ArtifactStore](ALLM.Pipeline.ArtifactStore.md): The shared wrapper over `ALLM.Pipeline.Artifacts` — stores and retrieves
pipeline artifacts (HTML, JSON, extracted text) through the configured
adapter, and returns a URL reference for the PostgreSQL `step_log`.
- [ALLM.Pipeline.Artifacts](ALLM.Pipeline.Artifacts.md): Storage behaviour for pipeline artifacts — the heavy bodies a `step_logs` row
deliberately does not carry (scraped HTML, extracted PDF text, decision JSON,
the per-step LLM-call envelope).
- [ALLM.Pipeline.Artifacts.Dynamo](ALLM.Pipeline.Artifacts.Dynamo.md): DynamoDB client for artifact storage.
- [ALLM.Pipeline.Artifacts.Filesystem](ALLM.Pipeline.Artifacts.Filesystem.md): Local-disk `ALLM.Pipeline.Artifacts` adapter — the one that lets a fresh
clone run pipelines with **zero cloud infrastructure**.
- [ALLM.Pipeline.Artifacts.Memory](ALLM.Pipeline.Artifacts.Memory.md): In-VM `ALLM.Pipeline.Artifacts` adapter — artifacts live in an `Agent` and
die with it.
- [ALLM.Pipeline.Artifacts.S3](ALLM.Pipeline.Artifacts.S3.md): S3 `ALLM.Pipeline.Artifacts` adapter — the large-tier backend for artifacts
that do not fit a DynamoDB item.
- [ALLM.Pipeline.Artifacts.Tiered](ALLM.Pipeline.Artifacts.Tiered.md): A size-routing `ALLM.Pipeline.Artifacts` adapter — small artifacts go to one
backend, large ones to another. It is what makes the tier list an adapter
*choice* rather than the hard-coded S3 residue `ALLM.Pipeline.ArtifactStore`
used to carry (architecture §3.6, §2.7).
- [ALLM.Pipeline.Config](ALLM.Pipeline.Config.md): Resolves the host-supplied collaborators the package cannot name directly.
- [ALLM.Pipeline.Context](ALLM.Pipeline.Context.md): Pipeline execution context passed to each step — and, since Phase 4, to every
`ALLM.Pipeline` escape-hatch body and `FanOut.reduce/5` fold function as well.
- [ALLM.Pipeline.Dsl](ALLM.Pipeline.Dsl.md): The construct macros behind `use ALLM.Pipeline`, plus the compile-time
validators.
- [ALLM.Pipeline.Dsl.Item](ALLM.Pipeline.Dsl.Item.md): One item's outcome from a `fan_out`, and the carrier for per-item lineage.
- [ALLM.Pipeline.Dsl.Resource](ALLM.Pipeline.Dsl.Resource.md): One declared `resource` — a handle acquired once per run and released before
the terminal write.
- [ALLM.Pipeline.Dsl.Runtime](ALLM.Pipeline.Dsl.Runtime.md): The interpreter the generated `run/1` calls.
- [ALLM.Pipeline.Dsl.Stage](ALLM.Pipeline.Dsl.Stage.md): One compiled stage of an `ALLM.Pipeline` declaration.
- [ALLM.Pipeline.Encodable](ALLM.Pipeline.Encodable.md): The single implementation of "make an arbitrary term safe for a jsonb column".
- [ALLM.Pipeline.Executor](ALLM.Pipeline.Executor.md): Base utilities for executing pipeline steps.
- [ALLM.Pipeline.FanOut](ALLM.Pipeline.FanOut.md): The canonical home for the fan-out safety rule, plus the one helper
every `Task.async_stream` site shares.
- [ALLM.Pipeline.LLM](ALLM.Pipeline.LLM.md): The seam through which the package calls a host's LLM engine.
- [ALLM.Pipeline.LLMCallLog](ALLM.Pipeline.LLMCallLog.md): Per-step collector for full LLM-call inputs/outputs.
- [ALLM.Pipeline.LLMStep](ALLM.Pipeline.LLMStep.md): A macro for the LLM-calling half of an `ALLM.Pipeline.Step`.
- [ALLM.Pipeline.Lifecycle](ALLM.Pipeline.Lifecycle.md): The run lifecycle guard, as ONE implementation with two consumers.
- [ALLM.Pipeline.Lock](ALLM.Pipeline.Lock.md): Serialization guard for pipeline runs — the abstraction behind "don't let two
runs of the same pipeline overlap on the shared database".
- [ALLM.Pipeline.Lock.Advisory](ALLM.Pipeline.Lock.Advisory.md): Postgres session-advisory-lock implementation of `ALLM.Pipeline.Lock`.
- [ALLM.Pipeline.Lock.Noop](ALLM.Pipeline.Lock.Noop.md): No-op `ALLM.Pipeline.Lock` — the current default.
- [ALLM.Pipeline.Metrics](ALLM.Pipeline.Metrics.md): Record and query normalized pipeline metrics (the found → mapped → processed funnel).
Emission point: a pipeline calls `record/3` at the SAME place it calls
`PipelineRun.complete/2` (the sole completer for umbrella/borrowed runs).

- [ALLM.Pipeline.PipelineMetric](ALLM.Pipeline.PipelineMetric.md): One normalized metrics row for a pipeline run: the found → mapped → processed
funnel (plus skipped/failed) for a single entity type. See the "Metric semantics"
section of steering/PIPELINE_METRICS_DASHBOARD.md.

- [ALLM.Pipeline.PipelineRun](ALLM.Pipeline.PipelineRun.md): Represents a single execution of a pipeline.
- [ALLM.Pipeline.Query](ALLM.Pipeline.Query.md): The package-owned, **read-only** query facade for host UIs and host contexts.
- [ALLM.Pipeline.Registry](ALLM.Pipeline.Registry.md): The host's one declaration of how it wires the framework, and of the domain
knowledge the framework must not carry itself.
- [ALLM.Pipeline.Schema](ALLM.Pipeline.Schema.md): A macro for defining Input/Output schemas with reduced boilerplate.
- [ALLM.Pipeline.Schema.JsonSchema](ALLM.Pipeline.Schema.JsonSchema.md): Derives an OpenAI **strict-mode** JSON schema from an `ALLM.Pipeline.Schema`
declaration.
- [ALLM.Pipeline.Step](ALLM.Pipeline.Step.md): Behavior for pipeline steps with strongly-typed inputs and outputs.
- [ALLM.Pipeline.StepLog](ALLM.Pipeline.StepLog.md): Persists step execution logs to PostgreSQL for observability and lineage tracking.
- [ALLM.Pipeline.Store](ALLM.Pipeline.Store.md): Persistence behaviour for **runs and steps** — the observability record a
pipeline leaves behind.
- [ALLM.Pipeline.Store.Ecto](ALLM.Pipeline.Store.Ecto.md): PostgreSQL implementation of `ALLM.Pipeline.Store`, and the adapter that
ships the package's run/step schemas.
- [ALLM.Pipeline.Telemetry](ALLM.Pipeline.Telemetry.md): The `[:allm_pipeline, …]` telemetry event contract, plus thin `emit_*`
helpers wrapping `:telemetry.execute/3`.
- [ALLM.Pipeline.Text](ALLM.Pipeline.Text.md): Batteries-included text handling for pipeline persistence: scrubbing text so
Postgres will accept it, whitespace normalization, and head+tail truncation
for oversized artifact bodies.

## Mix Tasks

- [mix allm_pipeline.names](Mix.Tasks.AllmPipeline.Names.md): Emits the installed `ALLM.Pipeline.Registry`'s per-pipeline metadata as a
machine-readable list of names.
- [mix allm_pipeline.nilability](Mix.Tasks.AllmPipeline.Nilability.md): Reports `ALLM.Pipeline.Schema`'s narrow nilability rule across every compiled
schema module in the current project.

