Search documentation of phoenix_kit

# PhoenixKit.Modules.Storage (phoenix_kit v1.7.87)

Copy Markdown View Source

Storage context for managing files, buckets, and dimensions.

Provides a distributed file storage system with support for multiple storage providers (local filesystem, AWS S3, Backblaze B2, Cloudflare R2) with automatic redundancy and failover capabilities.

## Features

- Multi-location storage with configurable redundancy (1-5 copies)
- Support for local, S3, B2, and R2 storage providers
- Automatic variant generation for images and videos
- Priority-based storage selection
- Built-in usage tracking and statistics
- PostgreSQL-backed file registry

## Module Status

This module is always enabled and cannot be disabled. It provides core functionality for file management across PhoenixKit.

## Summary

### Functions

calculate_bucket_free_space(bucket)

Calculates free space for a bucket.

calculate_bucket_usage(bucket_uuid)

Calculates storage usage for a bucket in MB.

calculate_user_file_checksum(user_uuid, file_checksum)

Calculates user-specific file checksum (salted with user_uuid).

change_bucket(bucket, attrs \\ %{})

Returns an `%Ecto.Changeset{}` for tracking bucket changes.

change_dimension(dimension, attrs \\ %{})

Returns an `%Ecto.Changeset{}` for tracking dimension changes.

change_file(file, attrs \\ %{})

Returns an `%Ecto.Changeset{}` for tracking file changes.

change_file_instance(instance, attrs \\ %{})

Returns an `%Ecto.Changeset{}` for tracking file instance changes.

count_orphaned_files()

Returns the count of orphaned files.

create_bucket(attrs \\ %{})

Creates a new bucket.

create_dimension(attrs \\ %{})

Creates a new dimension.

create_directory(path)

Creates a directory if it doesn't exist.

create_file(attrs \\ %{})

Creates a new file record.

create_file_instance(attrs \\ %{})

Creates a new file instance.

create_file_locations_for_instance(file_instance_uuid, bucket_uuids, file_path)

Creates file locations for a file instance across specified buckets.

delete_bucket(bucket)

Deletes a bucket.

delete_dimension(dimension)

Deletes a dimension.

delete_file(file)

Deletes a file.

delete_file_completely(file)

Deletes a file completely - physical data from all storage buckets and database record.

delete_file_data(file)

Deletes file data from all storage buckets for all variants.

delete_file_instance(instance)

Deletes a file instance.

ensure_default_bucket_exists()

Ensures at least one default bucket exists.

file_exists?(file)

Checks if a file exists in storage.

file_orphaned?(file_uuid)

Returns true if the given file UUID is not referenced by any known entity.

find_orphaned_files(opts \\ [])

Returns a list of orphaned files (files not referenced by any known entity).

get_absolute_path()

Gets the absolute path for local storage.

get_auto_generate_variants()
get_bucket(id)

Gets a single bucket by ID.

get_bucket_by_name(name)

Gets a bucket by name.

get_config()

Gets the current storage configuration.

get_default_path()

Gets the default storage path for local file uploads (relative path).

get_dimension(id)

Gets a single dimension by ID.

get_dimension_by_name(name)

Gets a dimension by name.

get_file(id)

Gets a single file by ID.

get_file_by_checksum(file_checksum)

Gets a file by its original content checksum (file_checksum).

get_file_by_user_checksum(user_file_checksum)

Gets a file by its user-specific checksum.

get_file_instance(id)

Gets a single file instance by ID.

get_file_instance_bucket_uuids(file_instance_uuid)

Gets the bucket UUIDs where a file instance is stored.

get_file_instance_by_name(file_uuid, variant_name)

Gets a file instance by file UUID and variant name.

get_public_url(file)

Gets a public URL for a file.

get_public_url_by_uuid(file_uuid)

Gets a public URL for a file by file ID.

get_public_url_by_uuid(file_uuid, variant_name)

Gets a public URL for a specific file variant by file ID.

get_public_url_by_variant(file, variant_name)

Gets a public URL for a specific file variant.

list_buckets()

Returns a list of all storage buckets, ordered by priority.

list_dimensions()

Returns a list of all dimensions, ordered by size (width x height).

list_dimensions_for_type(file_type)

Returns enabled dimensions for a specific file type.

list_enabled_buckets()

Gets enabled buckets, ordered by priority.

list_file_instances(file_uuid)

Returns a list of file instances for a given file.

list_files(opts \\ [])

Returns a list of files, optionally filtered by bucket.

module_enabled?()

Checks if the Storage module is enabled.

queue_file_cleanup(file_uuids)

Queues a list of file UUIDs for orphan cleanup via Oban.

repair_storage_module()

Repairs the storage module by resetting configuration to defaults.

reset_dimensions_to_defaults()

Resets all dimensions to default seeded values. Deletes all current dimensions and recreates the 8 default ones.

reset_settings_to_defaults()

Resets storage settings to their default values.

retrieve_file(file_uuid)

Retrieves a file from storage by file UUID.

retrieve_file_by_hash(hash)

Retrieves a file by its hash.

store_file(source_path, opts \\ [])

Stores a file in the storage system.

store_file_in_buckets(source_path, file_type, user_uuid, file_checksum, ext, original_filename \\ nil)

Stores a file in buckets with hierarchical path structure.

update_bucket(bucket, attrs)

Updates a bucket.

update_default_path(relative_path)

Updates the default storage path.

update_dimension(dimension, attrs)

Updates a dimension.

update_file(file, attrs)

Updates a file.

update_file_instance(instance, attrs)

Updates a file instance.

update_instance_status(instance, status)

Updates a file instance's processing status.

update_instance_with_file_info(instance, file_path, dimensions \\ nil)

Updates a file instance with file information after processing.

validate_and_normalize_path(path)

Validates and normalizes a storage path.

## Functions

# calculate_bucket_free_space(bucket)

Calculates free space for a bucket.

For local storage, checks actual disk space. For cloud storage, returns the configured max_size_mb minus usage.

# calculate_bucket_usage(bucket_uuid)

Calculates storage usage for a bucket in MB.

Returns total size of all files stored in this bucket by summing up all file instances that have locations in this bucket.

# calculate_user_file_checksum(user_uuid, file_checksum)

Calculates user-specific file checksum (salted with user_uuid).

This creates a unique checksum per user+file combination for duplicate detection, while preserving the original file checksum for popularity queries.

## Parameters

- user_uuid: The user UUID
- file_checksum: The SHA256 checksum of the file content

## Returns

String representing the SHA256 checksum of "user_uuid + file_checksum"

# change_bucket(bucket, attrs \\ %{})

Returns an `%Ecto.Changeset{}` for tracking bucket changes.

# change_dimension(dimension, attrs \\ %{})

Returns an `%Ecto.Changeset{}` for tracking dimension changes.

# change_file(file, attrs \\ %{})

Returns an `%Ecto.Changeset{}` for tracking file changes.

# change_file_instance(instance, attrs \\ %{})

Returns an `%Ecto.Changeset{}` for tracking file instance changes.

# count_orphaned_files()

Returns the count of orphaned files.

# create_bucket(attrs \\ %{})

Creates a new bucket.

## Examples

    iex> create_bucket(%{name: "Local Storage", provider: "local"})
    {:ok, %Bucket{}}

    iex> create_bucket(%{name: nil})
    {:error, %Ecto.Changeset{}}

# create_dimension(attrs \\ %{})

Creates a new dimension.

## Examples

    iex> create_dimension(%{name: "thumbnail", width: 150, height: 150})
    {:ok, %Dimension{}}

    iex> create_dimension(%{name: nil})
    {:error, %Ecto.Changeset{}}

# create_directory(path)

Creates a directory if it doesn't exist.

# create_file(attrs \\ %{})

Creates a new file record.

This only creates the database record. Use `store_file/4` to actually store the file data in storage buckets.

# create_file_instance(attrs \\ %{})

Creates a new file instance.

# create_file_locations_for_instance(file_instance_uuid, bucket_uuids, file_path)

Creates file locations for a file instance across specified buckets.

Returns `{:ok, locations}` on success or `{:error, :file_locations_failed, errors}` if any insertions fail.

## Parameters

- `file_instance_uuid` - The UUID of the file instance
- `bucket_uuids` - List of bucket UUIDs to create locations for
- `file_path` - The storage path for the file

## Examples

    iex> create_file_locations_for_instance(instance_uuid, [bucket_uuid], "path/to/file")
    {:ok, [%FileLocation{}]}

    iex> create_file_locations_for_instance(instance_uuid, [invalid_bucket], "path")
    {:error, :file_locations_failed, [{bucket_uuid, changeset}]}

# delete_bucket(bucket)

Deletes a bucket.

## Examples

    iex> delete_bucket(bucket)
    {:ok, %Bucket{}}

    iex> delete_bucket(bucket)
    {:error, %Ecto.Changeset{}}

# delete_dimension(dimension)

Deletes a dimension.

## Examples

    iex> delete_dimension(dimension)
    {:ok, %Dimension{}}

    iex> delete_dimension(dimension)
    {:error, %Ecto.Changeset{}}

# delete_file(file)

Deletes a file.

This only removes the database record. Use `delete_file_data/1` to remove the actual file data from storage buckets.

# delete_file_completely(file)

Deletes a file completely - physical data from all storage buckets and database record.

## Examples

    iex> delete_file_completely(file)
    {:ok, %File{}}

# delete_file_data(file)

Deletes file data from all storage buckets for all variants.

# delete_file_instance(instance)

Deletes a file instance.

# ensure_default_bucket_exists()

Ensures at least one default bucket exists.

If no buckets exist, creates a default local storage bucket.

## Returns

- `{:created, bucket}` - If a new bucket was created
- `:exists` - If buckets already exist

## Examples

    iex> ensure_default_bucket_exists()
    {:created, %Bucket{name: "Local Storage"}}

    iex> ensure_default_bucket_exists()
    :exists

# file_exists?(file)

Checks if a file exists in storage.

# file_orphaned?(file_uuid)

Returns true if the given file UUID is not referenced by any known entity.

# find_orphaned_files(opts \\ [])

Returns a list of orphaned files (files not referenced by any known entity).

## Options

- `:limit` - Maximum number of results
- `:offset` - Number of results to skip

# get_absolute_path()

Gets the absolute path for local storage.

# get_auto_generate_variants()

# get_bucket(id)

Gets a single bucket by ID.

Returns `nil` if bucket does not exist.

# get_bucket_by_name(name)

Gets a bucket by name.

# get_config()

Gets the current storage configuration.

# get_default_path()

Gets the default storage path for local file uploads (relative path).

Returns the configured relative path or the default "priv/uploads" if not set.

# get_dimension(id)

Gets a single dimension by ID.

# get_dimension_by_name(name)

Gets a dimension by name.

# get_file(id)

Gets a single file by ID.

# get_file_by_checksum(file_checksum)

Gets a file by its original content checksum (file_checksum).

This can find files uploaded by any user with the same content. Useful for popularity queries.

# get_file_by_user_checksum(user_file_checksum)

Gets a file by its user-specific checksum.

This checks for duplicates for a specific user.

# get_file_instance(id)

Gets a single file instance by ID.

# get_file_instance_bucket_uuids(file_instance_uuid)

Gets the bucket UUIDs where a file instance is stored.

Returns a list of bucket UUIDs from the file_locations for the given file instance.

# get_file_instance_by_name(file_uuid, variant_name)

Gets a file instance by file UUID and variant name.

# get_public_url(file)

Gets a public URL for a file.

# get_public_url_by_uuid(file_uuid)

Gets a public URL for a file by file ID.

Convenience function that fetches the file and returns its URL.

## Examples

    iex> get_public_url_by_uuid("018e3c4a-9f6b-7890-abcd-ef1234567890")
    "https://cdn.example.com/12/a1/a1b2c3d4e5f6/a1b2c3d4e5f6_original.jpg"

    iex> get_public_url_by_uuid("invalid-uuid")
    nil

# get_public_url_by_uuid(file_uuid, variant_name)

Gets a public URL for a specific file variant by file ID.

## Examples

    iex> get_public_url_by_uuid("018e3c4a-9f6b-7890-abcd-ef1234567890", "thumbnail")
    "https://cdn.example.com/12/a1/a1b2c3d4e5f6/a1b2c3d4e5f6_thumbnail.jpg"

# get_public_url_by_variant(file, variant_name)

Gets a public URL for a specific file variant.

## Variants

For images: "original", "thumbnail", "small", "medium", "large" For videos: "original", "360p", "720p", "1080p", "video_thumbnail"

## Examples

    iex> get_public_url_by_variant(file, "thumbnail")
    "https://cdn.example.com/12/a1/a1b2c3d4e5f6/a1b2c3d4e5f6_thumbnail.jpg"

    iex> get_public_url_by_variant(file, "medium")
    "https://cdn.example.com/12/a1/a1b2c3d4e5f6/a1b2c3d4e5f6_medium.jpg"

# list_buckets()

Returns a list of all storage buckets, ordered by priority.

# list_dimensions()

Returns a list of all dimensions, ordered by size (width x height).

# list_dimensions_for_type(file_type)

Returns enabled dimensions for a specific file type.

# list_enabled_buckets()

Gets enabled buckets, ordered by priority.

# list_file_instances(file_uuid)

Returns a list of file instances for a given file.

# list_files(opts \\ [])

Returns a list of files, optionally filtered by bucket.

## Options

- `:bucket_uuid` - Filter by bucket UUID
- `:limit` - Maximum number of results
- `:offset` - Number of results to skip
- `:order_by` - Ordering (default: `[desc: :inserted_at]`)

# module_enabled?()

Checks if the Storage module is enabled.

This module is always enabled and cannot be disabled.

## Examples

    iex> PhoenixKit.Modules.Storage.module_enabled?()
    true

# queue_file_cleanup(file_uuids)

Queues a list of file UUIDs for orphan cleanup via Oban.

Each file is scheduled for deletion after a 60-second delay to protect against race conditions (another entity may reference the file). Only files that are still orphaned at job execution time will be deleted.

# repair_storage_module()

Repairs the storage module by resetting configuration to defaults.

This is a safe, non-destructive operation that:

1. Creates a default local bucket if no buckets exist
2. Resets dimensions to 8 defaults (4 image + 4 video)
3. Resets storage settings to recommended defaults

All existing files are preserved.

## Returns

- `{:ok, repairs}` - List of repairs performed
- `{:error, reason}` - If repair failed

## Examples

    iex> repair_storage_module()
    {:ok, [{:bucket_created, "Local Storage"}, {:dimensions_reset, 8}, {:settings_reset, 3}]}

# reset_dimensions_to_defaults()

Resets all dimensions to default seeded values. Deletes all current dimensions and recreates the 8 default ones.

# reset_settings_to_defaults()

Resets storage settings to their default values.

Resets:

- `storage_redundancy_copies` to "1"
- `storage_auto_generate_variants` to "true"
- `storage_default_bucket_uuid` to nil

## Returns

- `:ok`

# retrieve_file(file_uuid)

Retrieves a file from storage by file UUID.

Will try buckets in priority order until the file is found.

# retrieve_file_by_hash(hash)

Retrieves a file by its hash.

# store_file(source_path, opts \\ [])

Stores a file in the storage system.

This will:

1. Store the file in multiple buckets based on redundancy settings
2. Generate variants if enabled
3. Create database records for the file and its variants

## Options

- `:filename` - Original filename (required)
- `:content_type` - MIME type (required)
- `:size_bytes` - File size in bytes (required)
- `:user_uuid` - User UUID who owns the file
- `:metadata` - Additional metadata map

# store_file_in_buckets(source_path, file_type, user_uuid, file_checksum, ext, original_filename \\ nil)

Stores a file in buckets with hierarchical path structure.

## Path Structure

Files are stored using the pattern: `{user_uuid[0..1]}/{hash[0..1]}/{full_hash}/{full_hash}_{variant}.{format}`

## Examples

User ID: "12345678" File hash: "a1b2c3d4e5f6..." Original: "12/a1/a1b2c3d4e5f6/a1b2c3d4e5f6_original.jpg" Thumbnail: "12/a1/a1b2c3d4e5f6/a1b2c3d4e5f6_thumbnail.jpg"

# update_bucket(bucket, attrs)

Updates a bucket.

## Examples

    iex> update_bucket(bucket, %{name: "New Name"})
    {:ok, %Bucket{}}

    iex> update_bucket(bucket, %{name: nil})
    {:error, %Ecto.Changeset{}}

# update_default_path(relative_path)

Updates the default storage path.

# update_dimension(dimension, attrs)

Updates a dimension.

## Examples

    iex> update_dimension(dimension, %{name: "New Name"})
    {:ok, %Dimension{}}

    iex> update_dimension(dimension, %{name: nil})
    {:error, %Ecto.Changeset{}}

# update_file(file, attrs)

Updates a file.

# update_file_instance(instance, attrs)

Updates a file instance.

# update_instance_status(instance, status)

Updates a file instance's processing status.

# update_instance_with_file_info(instance, file_path, dimensions \\ nil)

Updates a file instance with file information after processing.

# validate_and_normalize_path(path)

Validates and normalizes a storage path.

Returns `{:ok, relative_path}` if valid, or error tuple if invalid.