Stores rendered sitemap XML on Gamend.Storage so a request serves bytes
instead of rebuilding them.
A large sitemap is ~19 MB and ~5,000 URLs, 90% of it the hreflang alternates every URL carries. Building one walks the host's content, resolves a date per key and interleaves an alternate per locale per URL — and a controller does that on every request, for a public endpoint anyone can loop on. Fifty children is roughly a gigabyte of generation per full crawl.
Keyed by a content signature, not by name alone
The obvious version — precompute at restamp time — goes stale the moment
something changes that mix gamend.sitemap.lastmod does not stamp: a new
blog post, a new page, a language gaining content. Those move the sitemap
without moving the manifest.
So the key carries a signature of everything the output depends on. A change
to any of it lands on a key that does not exist yet and regenerates once; the
old object is simply orphaned, and swept by prune/0. That makes the cache
self-healing: forgetting to warm it costs one slow request, never a wrong
sitemap.
The signature is deliberately cheap — a File.stat of the manifest rather
than a hash of its thousands of entries, because it is computed on every
request, and it doubles as the ETag.
What the host contributes
Core knows the inputs every host shares — the manifest's stamp, the endpoint
URL, the locale set. It cannot know the rest, so c:signature_inputs/0 on
the configured GamendWeb.Sitemap.Source returns them: the counts and
versions that decide how many URLs exist.
Anything the rendered XML depends on belongs in that list. Include a version number the host bumps by hand, too — pages decided by code rather than data change the output and nothing derived from content would notice.
Stored gzipped
Uncompressed, fifty children would be about a gigabyte on the volume; XML of
this shape compresses roughly ten to one. Every crawler sends
accept-encoding: gzip, so the common path also skips decompression
entirely — the stored bytes go straight out.
Summary
Functions
Gzipped bytes for name, generating and storing them on a miss.
The cache key for name under the current signature.
Deletes every cached sitemap whose signature is not the current one.
Renders name through the cache and sends it.
The signature every cache key is scoped by, and the ETag every response carries.
Functions
Gzipped bytes for name, generating and storing them on a miss.
generate returns the iodata body, and is only called on a miss — building
it eagerly and passing the result would leave the whole cost in place and
cache only the write.
A storage failure is logged and downgraded to a miss: serving an uncached sitemap is better than serving none.
The cache key for name under the current signature.
name is the sitemap's own name — "index", "pages", or whatever the host
calls a child.
@spec prune() :: :ok
Deletes every cached sitemap whose signature is not the current one.
Nothing depends on this — a stale object is unreachable, not wrong — so it is housekeeping rather than part of the request path.
@spec serve(Plug.Conn.t(), String.t(), (-> iodata()), keyword()) :: Plug.Conn.t()
Renders name through the cache and sends it.
The signature doubles as the ETag, so a crawler that already has the file gets a 304 and the body is never sent at all — which is the cheapest possible answer to the request a crawler makes most often.
Bytes are stored gzipped, which is what every crawler asks for anyway; only a client that does not accept gzip pays for the decompression.
Options: :cache_control (default one day — the sitemap changes when content
does, and the ETag makes a repeat fetch a 304 either way).
@spec signature() :: String.t()
The signature every cache key is scoped by, and the ETag every response carries.
Covers each input that changes the rendered XML: the manifest (mtime and
size — a restamp moves both), the endpoint URL every <loc> is built from,
the locale set that decides the alternates per URL, and whatever the host's
GamendWeb.Sitemap.Source.signature_inputs/0 adds.