GamendWeb.Sitemap.Lastmod (gamend_web v1.0.1215)

Copy Markdown View Source

Truthful <lastmod> dates for sitemap entries, from content hashing.

Why not file mtimes or git dates

Both are dates of events that touched a file, not dates the content changed, and search engines discount lastmod from sites whose dates move when nothing did.

A checkout sets every mtime to the moment of the checkout, so a redeploy would claim the whole site changed. Git commit dates survive that, but only until a tool rewrites files wholesale — a line-ending change, a re-export, a column reshuffle — at which point every page in the commit claims to have changed and none of them did.

So the date comes from the content itself. Each page contributes the text a visitor would actually see; that is hashed, and the stored date moves only when the hash does. Reformatting the source file is invisible here, because the reformatting never reaches the hashed value — the caller passes parsed content, not raw bytes.

The manifest

A JSON file, committed to the repository, mapping page key to hash and date:

{
  "vocabulary/polish/food": {"hash": "9f86d081...", "date": "2026-07-14"}
}

It is regenerated by mix gamend.sitemap.lastmod and read at request time. Regeneration rebuilds the map from the current entries, so a page that no longer exists drops out rather than accumulating forever.

Bootstrapping

The first run has no prior hashes, so every page looks new and would be stamped with today — the exact lie this module exists to avoid. To bootstrap honestly, an entry may carry a :seed_date: a date the caller has independent reason to believe (a git commit date for the backing file, say). It is used only when the key is absent from the manifest. Every run after that is driven by the hash, so a coarse seed cannot keep distorting things.

Summary

Types

One page to date.

Functions

The stored date for a key, or nil when absent.

The manifest's on-disk form, for writing or for comparing in --check.

Hash of a page's content.

The newest date among keys, or nil when none are known.

Reads a manifest from disk.

Writes a manifest, sorted and pretty-printed so diffs stay readable.

Rebuilds the manifest from entries.

Types

entry()

@type entry() :: %{
  :key => String.t(),
  :content => [String.t()],
  optional(:seed_date) => String.t() | nil
}

One page to date.

content is the page's meaningful text, in display order, as a flat list of strings — parsed values, never raw file bytes. seed_date is an ISO 8601 date used only when the key is new to the manifest.

manifest()

@type manifest() :: %{required(String.t()) => %{required(String.t()) => String.t()}}

Functions

date(manifest, key)

@spec date(manifest(), String.t()) :: String.t() | nil

The stored date for a key, or nil when absent.

encode(manifest)

@spec encode(manifest()) :: String.t()

The manifest's on-disk form, for writing or for comparing in --check.

hash(content)

@spec hash([String.t()]) :: String.t()

Hash of a page's content.

Parts are length-prefixed before hashing so that no regrouping of the same characters collides: ["ab", "c"] and ["a", "bc"] hash differently. Line endings are normalised and each part trimmed, so a CRLF rewrite or trailing-whitespace churn in the source does not move the hash.

latest(manifest, keys)

@spec latest(manifest(), [String.t()]) :: String.t() | nil

The newest date among keys, or nil when none are known.

This is what a sitemap index entry carries: the child's lastmod is the most recent change anywhere inside it.

load(path)

@spec load(Path.t()) :: manifest()

Reads a manifest from disk.

A missing file is an empty manifest, not an error: that is the state before the first generation, and a sitemap without dates is valid.

save(path, manifest)

@spec save(Path.t(), manifest()) :: :ok | {:error, File.posix()}

Writes a manifest, sorted and pretty-printed so diffs stay readable.

stamp(manifest, entries, today)

@spec stamp(manifest(), [entry()], Date.t()) :: {manifest(), [String.t()]}

Rebuilds the manifest from entries.

Returns {manifest, changed_keys}. A key keeps its stored date while its hash is unchanged; otherwise it takes today — or its :seed_date, when the key is new and one was supplied. changed_keys lists what actually moved, which is what a change notifier (IndexNow) wants to submit; a seeded first sighting is not reported as a change, because nothing changed, we simply had not looked before.