The s3:// variant store: one object per variant in a bucket.
This is the store for a deployment that has object storage — the cache
outlives the container, every node reads what any node rendered, and,
because an object can be handed to a client directly, AP_SERVE_MODE=redirect
becomes reachable. That last part is not a bonus: redirect is the documented
default (docs/audio-proxy-api-v1.md §5), and until this backend existed no
shipped store could presign, so every deployment served cached bytes through
the BEAM.
Layout: the cache key is the object key
No fan-out. AudioProxy.VariantStore.Local splits a key into
ab/cd/abcd… because a directory holding a million entries is a filesystem
problem; a bucket holding a million objects is not one, and S3 partitions by
key prefix on its own. A flat key is also what makes the reserved probe
prefix below unambiguous.
Keys are still checked against the cache-key shape before they name an
object. Not for traversal — an object key has no .. to follow — but so
that nothing outside AudioProxy.CacheKey's alphabet can be written into
the bucket, which is what keeps AudioProxy.VariantStore.S3 and
AudioProxy.Config's boot probe out of each other's key space.
A PUT is the commit point
There is no staging, because there is nothing to stage: an object does not
exist until its upload completes, and AudioProxy.S3.put_stream/4 aborts a
multipart upload on every failure path it can see. So the local backend's
tmp/ dance and its boot-time sweep have no analogue here, and
atomic-or-absent is a property of the protocol rather than of this module.
Metadata rides on the object
Content-Type and Cache-Control are the object's own headers. That is
what makes a redirect correct: the client fetches the object from the store
with no proxy in the path, and must receive the same headers a proxied HIT
would have sent. The seam's etag cannot be one — S3 computes an object's
ETag itself — so it travels as x-amz-meta-etag and is read back off the
HEAD.
An object missing any of the three — or carrying it empty — is reported as a
miss rather than served with a guess, by head/1 and get_stream/2 alike.
Both, deliberately: a callback that answered from any object under the key
while the other called it absent would be the two disagreeing about whether
the key is stored, and get_stream/2 pays a head/1 to avoid it. So
head/1 answering {:ok, _} means this module wrote the object, whole; a
stray object under a 64-hex key is not a variant to either callback.
Failures are misses, and the store's failures are the operator's
The read side of the seam has two error values — :not_found and
:invalid_range — so every other way S3 can fail arrives at a caller that
can only render. (put_stream/3 is wider: it also answers :invalid_key,
and hands back whatever AudioProxy.S3 returned, because the tee that calls
it reports failures rather than recovering from them.) Collapsing reads to a
miss is the right answer: a render produces correct bytes whatever the cache
did, and failing a request because the cache is unreachable would turn a
degraded store into an outage.
It is not a silent answer. AudioProxy.Source.S3.classify/1 — the table
add-s3-source-backend established, reused here rather than restated — says
whether a failure was an ordinary miss, a misconfiguration, or an outage,
and anything that is not an ordinary miss is logged at warning with the
bucket and key. An operator reading "every request is a MISS" needs the line
that says the store answered 403.
Reused with exactly one row read differently, and classify/1 below is where
and why: a 404 means something different to a store the operator configured
than to a source the client named.
Write failures are different only in who hears about them:
AudioProxy.VariantStore.Tee already treats the write-back as best-effort
and reports [:audio_proxy, :variant_store, :write_failure], so this module
returns the error as data and lets the tee decide. A client receiving a
correct render is never failed because the cache could not keep it.