Storage manager for handling file operations with redundancy and failover.
This module coordinates file storage across multiple buckets with automatic redundancy, failover, and variant generation.
Where an object is read from (V204)
Reads, existence checks, public URLs and serving go to the buckets the
object's location rows name first (PhoenixKit.Modules.Storage.Locations),
in the usual order: local buckets, then by priority. Only when none of
them has it are the other enabled buckets tried, and a bucket found to
hold it that way is recorded as a location. Deletes still remove an
unreferenced key from every enabled bucket.
Where an object is written (V205)
By the storage profile of the file's library (store_file/2): its
buckets, their roles and write priorities, and its copy counts.
Summary
Functions
Deletes a file from all storage buckets.
Deletes the object at key from bucket only, never from any other
bucket. Storage.unlink_location/2 is what decides that nothing else on
that bucket needs it (G11).
Checks if a file exists in any storage bucket.
Returns file access info based on bucket access_type.
Returns the local filesystem path for a file if stored locally.
Whether bucket has the object at key: the reconciler's check that a
copy it made is really there before it unlinks another. A bucket that
raises does not have it.
Drops the cached list of enabled buckets, so the next read sees the database. Called whenever a bucket is created, updated or deleted. The cache only spares a query per file request; it expires on its own after five minutes, which is how long a bucket edit used to take to apply.
Gets a public URL for a file from the highest priority bucket that has it.
Replicates a file to specific target buckets.
Retrieves a file from storage with failover.
Stores a file across buckets.
Functions
Deletes a file from all storage buckets.
Deletes the object at key from bucket only, never from any other
bucket. Storage.unlink_location/2 is what decides that nothing else on
that bucket needs it (G11).
Checks if a file exists in any storage bucket.
Returns file access info based on bucket access_type.
Determines how a file should be served based on its storage location:
- Local files are always served directly
- Remote files depend on bucket's access_type setting
Returns
{:local, path}- File is local, serve directly from filesystem{:redirect, url}- Redirect to public URL (for public buckets){:signed_redirect, url}- Redirect to a short-lived signed URL (a public bucket, with:downloadgiven){:proxy, file_name}- Download and proxy through server (for private buckets){:error, :not_found}- File not found in any bucket
Options
:download- response headers (:disposition,:content_type) for a file that must download rather than render. A public bucket's plain object URL answers with the object's stored headers, so for such a file a public bucket hands out a signed URL that overrides them (PhoenixKit.Modules.Storage.Provider.signed_download_url/3), or is proxied when its provider cannot sign one.
Returns the local filesystem path for a file if stored locally.
This function checks enabled buckets with "local" provider and returns the direct path to the file if it exists. This allows serving files directly without copying to a temp file first.
Returns
{:ok, path}- File exists at the local path{:error, :not_local}- No local bucket contains the file
Whether bucket has the object at key: the reconciler's check that a
copy it made is really there before it unlinks another. A bucket that
raises does not have it.
Drops the cached list of enabled buckets, so the next read sees the database. Called whenever a bucket is created, updated or deleted. The cache only spares a query per file request; it expires on its own after five minutes, which is how long a bucket edit used to take to apply.
Gets a public URL for a file from the highest priority bucket that has it.
Replicates a file to specific target buckets.
Retrieves the file from any available bucket, then stores it to each target bucket. Returns the list of bucket UUIDs that succeeded.
Retrieves a file from storage with failover.
Tries each bucket in priority order until the file is found.
Stores a file across buckets.
Where it goes (V205)
By a storage profile: :profile (a StorageProfile with its buckets
preloaded, Storage.Profiles.for_library/1), or the Default profile
when none is given. :kind says what the object is: :original (an
original upload, the default) or :derived (a size, a tile, a render),
which picks the profile's copy count and the buckets whose stores
allows it. Buckets that are enabled, active in the profile and not over
their max_size_mb are written, primaries before replicas before
backups, each group by fixed write priority and then in random order,
up to the copy count.
An original fails unless at least the profile's min_copies_on_write
copies were written (what was written is removed again). The result
says whether every copy the profile wants was made (complete?); the
rest are made by the reconciler.
Options
:path_prefix- the object key:profile/:kind- see above:force_bucket_ids- exactly these buckets (every one that is enabled, in the order given), whatever any profile says:redundancy_copies/:priority_buckets- the pre-profile selection (the enabled buckets by priority, up to the count), kept for callers that ask for it
Returns
{:ok, file_result}-destination_path,bucket_ids(the buckets written),stored_in,successful_storages,complete?{:error, reason}- Failed to store file