Buckets.Cloud behaviour (buckets v1.0.0-rc.0)
Defines a cloud.
A cloud manages the movement of files and data between your application and remote buckets for persistent storage.
When used, it expects :otp_app
and :default_location
as options:
defmodule MyApp.Cloud do
use Buckets.Cloud,
otp_app: :my_app,
default_location: :local
end
Additional configuration is fetched from the application config, using a
combination of :otp_app
and the module that you defined. You may configure
many "locations" in your config, for setting up multi-cloud support:
config :my_app, MyApp.Cloud,
locations: [
local: [
strategy: Buckets.Strategy.Volume,
# configure this strategy...
],
gcs: [
strategy: Buckets.Strategy.GCS,
# configure this strategy...
],
us_east_1: [
strategy: Buckets.Strategy.S3,
# configure this strategy...
]
]
You may also specify config dynamically at runtime, using the :config
opt
where it is supported.
When building a multi-cloud application and persisting uploaded objects to a database, you should take care to store some indicator of the location config that was used to insert a particular object. Otherwise, you won't know how to fetch data for or manage that object the next time it is accessed.
Similarly, if you are building a dynamic-cloud application (say, so that your users can specify their own clouds for your application to use), you should store the configuration provided by the user. As objects are inserted to the cloud and stored in your database, add a mapping between the object and the user-provided config. If this config is ever deleted, objects managed by it will become inaccessible. Keep this in mind as you build features to let users update or delete their config.
Summary
Callbacks
Deletes a Buckets.Object
permanently.
Same as delete/1
but returns the object or raises if there is an error.
Inserts a Buckets.Object
or a file from a path into a bucket.
Same as insert/2
but returns the object or raises if there is an error.
Returns a map to be used as configuration for a LiveView live upload. The configuration
contains a signed URL that permits the upload to a remote bucket. Requires that an :uploader
is configured for the location that the file is being uploaded to.
Same as c:live_upload/1
but returns the upload config or raises if there is an error.
Loads the data for a Buckets.Object
, placing it in memory by default. It will load data
lazily, doing nothing if data is already present.
Same as c:load/1
but returns the object or raises if there is an error.
An overridable function that processes filenames before encoding them in a remote path.
Reads the data for Buckets.Object
, preferring local data first and fetching from
the remote bucket if needed.
Same as read/1
but returns the data or raises if there is an error.
An overridable function that specifies the temporary directory in which objects are stored.
Unloads the data for a Buckets.Object
. If the data is stored in a local file, the file will
be deleted.
Returns a SignedURL struct for a Buckets.Object
.
Same as url/1
but returns the SignedURL raises if there is an error.
Callbacks
delete(object)
@callback delete(object :: Buckets.Object.t()) :: {:ok, Buckets.Object.t()} | {:error, term()}
Deletes a Buckets.Object
permanently.
delete!(object)
@callback delete!(object :: Buckets.Object.t()) :: Buckets.Object.t()
Same as delete/1
but returns the object or raises if there is an error.
insert(object_or_path, opts)
@callback insert( object_or_path :: Buckets.Object.t() | String.t(), opts :: Keyword.t() ) :: {:ok, Buckets.Object.t()} | {:error, term()}
Inserts a Buckets.Object
or a file from a path into a bucket.
insert!(object_or_path, opts)
@callback insert!( object_or_path :: Buckets.Object.t() | String.t(), opts :: Keyword.t() ) :: Buckets.Object.t()
Same as insert/2
but returns the object or raises if there is an error.
live_upload(entry, opts)
@callback live_upload( entry :: Phoenix.LiveView.UploadEntry.t(), opts :: Keyword.t() ) :: {:ok, map()} | {:error, term()}
Returns a map to be used as configuration for a LiveView live upload. The configuration
contains a signed URL that permits the upload to a remote bucket. Requires that an :uploader
is configured for the location that the file is being uploaded to.
live_upload!(entry, opts)
@callback live_upload!( entry :: Phoenix.LiveView.UploadEntry.t(), opts :: Keyword.t() ) :: map()
Same as c:live_upload/1
but returns the upload config or raises if there is an error.
load(object, opts)
@callback load( object :: Buckets.Object.t(), opts :: Keyword.t() ) :: {:ok, Buckets.Object.t()} | {:error, term()}
Loads the data for a Buckets.Object
, placing it in memory by default. It will load data
lazily, doing nothing if data is already present.
Options
* `:to` - A location to store the loaded data to on disk, if this is preferred over
loading to memory. May be a path, `:tmp` (to load to the configured tmp dir), or
`{:tmp, path}` (to load to a path in the configured tmp dir).
* `:force` - If data is already present, first unload it with `c:unload/1` before loading
new data. Warning: if data is stored in a file, it will be deleted.
load!(object, opts)
@callback load!(object :: Buckets.Object.t(), opts :: Keyword.t()) :: Buckets.Object.t()
Same as c:load/1
but returns the object or raises if there is an error.
normalize_filename(filename)
An overridable function that processes filenames before encoding them in a remote path.
By default, replaces whitespace with "_"
and removes all non-alphanumeric characters.
read(object)
@callback read(object :: Buckets.Object.t()) :: {:ok, binary()} | {:error, term()}
Reads the data for Buckets.Object
, preferring local data first and fetching from
the remote bucket if needed.
read!(object)
@callback read!(object :: Buckets.Object.t()) :: binary()
Same as read/1
but returns the data or raises if there is an error.
tmp_dir()
@callback tmp_dir() :: String.t()
An overridable function that specifies the temporary directory in which objects are stored.
Defaults to System.tmp_dir!()
.
unload(object)
@callback unload(object :: Buckets.Object.t()) :: Buckets.Object.t()
Unloads the data for a Buckets.Object
. If the data is stored in a local file, the file will
be deleted.
url(object)
@callback url(object :: Buckets.Object.t()) :: Buckets.Object.t()
Returns a SignedURL struct for a Buckets.Object
.
url!(object, opts)
@callback url!(object :: Buckets.Object.t(), opts :: Keyword.t()) :: Buckets.Object.t()
Same as url/1
but returns the SignedURL raises if there is an error.