PhoenixKit.Modules.Storage.Workers.ChecksumBackfillJob (phoenix_kit v2.41.0)

Copy Markdown View Source

Recomputes the MD5 checksums the upload API used to record (G12).

file_checksum is the dedup key's input, and every upload path but UploadController hashed with SHA-256, so the same bytes uploaded through the API and through the media browser were never recognised as the same file. The controller hashes with SHA-256 now; this job finds the rows it stored before (a 32-hex-character checksum), reads each file's original, and records its SHA-256 and the matching user_file_checksum (Storage.calculate_user_file_checksum/3, library-aware).

A row whose uploader already has the same bytes in the same library would collide on the dedup index: it is left as it is (it keeps not deduping, which is what it did before). A file whose original cannot be read is left too. Either is marked in its metadata ("checksum_backfill"), so a later pass does not download it again. The update is guarded on the old checksum, so a row changed meanwhile (an image edit) is not overwritten.

Throttled and self-queuing like LocationBackfillJob: one batch per run, the next five seconds later, queued after boot and by the daily trash prune while any MD5 row is left.

Summary

Functions

Queues a pass when any file has an MD5 checksum. Never raises.

Runs a whole pass in the calling process; returns how many files were :updated, :duplicate (left, their uploader has the bytes already), :unreadable or :changed (edited meanwhile).

Functions

maybe_enqueue()

@spec maybe_enqueue() :: :queued | :nothing_to_do | :unavailable

Queues a pass when any file has an MD5 checksum. Never raises.

run_pass()

@spec run_pass() :: %{required(atom()) => non_neg_integer()}

Runs a whole pass in the calling process; returns how many files were :updated, :duplicate (left, their uploader has the bytes already), :unreadable or :changed (edited meanwhile).