The s3:// variant store: one object per variant in a bucket.
This is the store for a deployment that has object storage — the cache
outlives the container, every node reads what any node rendered, and,
because an object can be handed to a client directly, AP_SERVE_MODE=redirect
becomes reachable. That last part is not a bonus: redirect is the documented
default (docs/audio-proxy-api-v1.md §5), and until this backend existed no
shipped store could presign, so every deployment served cached bytes through
the BEAM.
Layout: the cache key is the object key
No fan-out. AudioProxy.VariantStore.Local splits a key into
ab/cd/abcd… because a directory holding a million entries is a filesystem
problem; a bucket holding a million objects is not one, and S3 partitions by
key prefix on its own. A flat key is also what makes the reserved probe
prefix below unambiguous.
Keys are still checked against the cache-key shape before they name an
object. Not for traversal — an object key has no .. to follow — but so
that nothing outside AudioProxy.CacheKey's alphabet can be written into
the bucket, which is what keeps AudioProxy.VariantStore.S3 and
AudioProxy.Config's boot probe out of each other's key space.
The store's own identity
Every request here — HEAD, multipart write-back, ranged read, presign — runs
under AudioProxy.S3's :store profile, as does AudioProxy.Config's boot
writability probe. With no AP_VARIANT_S3_* set that profile is the
source-side one and nothing about this module's behavior differs; with them
set, the store can live on another provider or answer to another principal,
and a source credential is never what writes a variant.
The presign matters most and is the least visible: a SigV4 signature covers the host, so a redirect-mode HIT signed with the source profile against a store on a different endpoint is not a degraded URL, it is an unverifiable one.
A PUT is the commit point
There is no staging, because there is nothing to stage: an object does not
exist until its upload completes, and AudioProxy.S3.put_stream/5 aborts a
multipart upload on every failure path it can see. So the local backend's
tmp/ dance and its boot-time sweep have no analogue here, and
atomic-or-absent is a property of the protocol rather than of this module.
Metadata rides on the object
Content-Type and Cache-Control are the object's own headers. That is
what makes a redirect correct: the client fetches the object from the store
with no proxy in the path, and must receive the same headers a proxied HIT
would have sent. The seam's etag cannot be one — S3 computes an object's
ETag itself — so it travels as x-amz-meta-etag and is read back off the
HEAD.
An object missing any of the three — or carrying it empty — is reported as a
miss rather than served with a guess, by head/1 and get_stream/2 alike.
Both, deliberately: a callback that answered from any object under the key
while the other called it absent would be the two disagreeing about whether
the key is stored, and get_stream/2 pays a head/1 to avoid it. So
head/1 answering {:ok, _} means this module wrote the object, whole; a
stray object under a 64-hex key is not a variant to either callback.
Failures are misses, and the store's failures are the operator's
The read side of the seam has two error values — :not_found and
:invalid_range — so every other way S3 can fail arrives at a caller that
can only render. (put_stream/3 is wider: it also answers :invalid_key,
and hands back whatever AudioProxy.S3 returned, because the tee that calls
it reports failures rather than recovering from them.) Collapsing reads to a
miss is the right answer: a render produces correct bytes whatever the cache
did, and failing a request because the cache is unreachable would turn a
degraded store into an outage.
It is not a silent answer. AudioProxy.Source.S3.classify/1 — the table
add-s3-source-backend established, reused here rather than restated — says
whether a failure was an ordinary miss, a misconfiguration, or an outage,
and anything that is not an ordinary miss is logged at warning with the
bucket and key. An operator reading "every request is a MISS" needs the line
that says the store answered 403.
Reused with exactly one row read differently, and classify/1 below is where
and why: a 404 means something different to a store the operator configured
than to a source the client named.
Write failures are different only in who hears about them:
AudioProxy.VariantStore.Tee already treats the write-back as best-effort
and reports [:audio_proxy, :variant_store, :write_failure], so this module
returns the error as data and lets the tee decide. A client receiving a
correct render is never failed because the cache could not keep it.