A size-routing ALLM.Pipeline.Artifacts adapter — small artifacts go to one
backend, large ones to another. It is what makes the tier list an adapter
choice rather than the hard-coded S3 residue ALLM.Pipeline.ArtifactStore
used to carry (architecture §3.6, §2.7).
The canonical wiring: DynamoDB for what fits an item, S3 for what does not.
config :allm_pipeline, ALLM.Pipeline.Artifacts,
impl: ALLM.Pipeline.Artifacts.Tiered
config :allm_pipeline, ALLM.Pipeline.Artifacts.Tiered,
small: ALLM.Pipeline.Artifacts.Dynamo,
large: ALLM.Pipeline.Artifacts.S3(A host declaring an ALLM.Pipeline.Registry writes those keys from
artifacts: {ALLM.Pipeline.Artifacts.Tiered, small: …, large: …} — see that
module's "The artifacts: tuple form".)
The routing decision is measured on POST-ENCODE bytes — this is the §2.7 fix
put/4 receives the payload already gzipped by ArtifactStore. It routes on
ALLM.Pipeline.Artifacts.Dynamo.encoded_size/1 — the base64-inflated,
as-stored size DynamoDB actually constrains — NOT the caller's raw content
length. A 640 KB HTML scrape that gzips to 40 KB therefore stays in the small
tier, where the old pre-encode store/4 branch wrongly sent it to the
(unimplemented) large tier and lost it.
threshold: is the post-encode ceiling for the small tier and defaults to
Dynamo.max_payload_bytes/0 — the DynamoDB item capacity — so small: Dynamo
with no explicit threshold routes exactly as Dynamo.fits_item?/1 does. It is
overridable (chiefly for tests, which want a small boundary without large
payloads).
Read dispatch
fetch/1 / delete/1 / exists?/1 route by URL, not by size. Each backing
adapter answers a URL it does not own with a structured "not mine" error
({:error, {:invalid_artifact_url, url}} for fetch, {:error, :invalid_url}
for delete, false for exists?), so this adapter tries the small tier and
falls through to the large one on that signal — adapter-agnostic, so it works
for any disjoint-scheme pair (dynamo:///s3:// in production,
memory:///file:// in tests) without hard-coding a scheme.