Magpie.Storage (Magpie v0.6.2)

Copy Markdown View Source

A small object-storage-style API backed by Dropbox.

Storage is the convenient entry point for common application operations; Magpie.Files remains available when Dropbox-specific controls are needed.

alias Magpie.Storage

{:ok, %Magpie.FileMetadata{}} =
  Storage.put(client, "/reports/today.json", {:binary, ~s({"ok":true})})

{:ok, contents} = Storage.get(client, "/reports/today.json")
{:ok, "/tmp/today.json"} =
  Storage.download(client, "/reports/today.json", "/tmp/today.json")

Upload sources are explicit:

  • {:file, path} streams a local file and automatically selects a single request or an upload session from its size;
  • {:binary, iodata} uploads in-memory content and makes the same choice;
  • {:stream, enumerable} always uses an upload session, buffering no more than one configured chunk between requests.

Paths are Dropbox paths, not S3 bucket/key pairs. Temporary download URLs expire according to Dropbox's rules (normally after four hours).

Summary

Functions

Copies a Dropbox file or folder to destination.

Like copy/4, but returns metadata directly and raises on failure.

Deletes a Dropbox file or folder and returns its final metadata.

Like delete/3, but returns metadata directly and raises on failure.

Deletes several keys concurrently while preserving input order and isolating failures.

Streams key to a local destination and returns that destination.

Like download/4, but returns the destination directly and raises on failure.

Returns true, false for a missing key, or an error tuple for other failures.

Downloads key into memory and returns its bytes.

Like get/3, but returns the bytes directly and raises on failure.

Returns all entries below prefix, following every cursor page.

Like list/3, but returns entries directly and raises on failure.

Creates a Dropbox folder at key.

Like mkdir/3, but returns metadata directly and raises on failure.

Moves a Dropbox file or folder to destination.

Like move/4, but returns metadata directly and raises on failure.

Uploads a file, binary/iodata value, or stream to key.

Like put/4, but raises on failure.

Uploads several {key, source} or {key, source, options} entries concurrently.

Returns typed Dropbox metadata for key.

Like stat/3, but returns metadata directly and raises on failure.

Returns a lazy stream over every entry below prefix.

Returns a one-use direct-upload URL for key.

Like upload_url/3, but returns the URL directly and raises on failure.

Returns a temporary direct-download URL for key.

Like url/3, but returns the URL directly and raises on failure.

Types

result(value)

@type result(value) :: {:ok, value} | {:error, Exception.t() | File.posix()}

source()

@type source() :: {:file, Path.t()} | {:binary, iodata()} | {:stream, Enumerable.t()}

Functions

copy(client, source, destination, opts \\ [])

Copies a Dropbox file or folder to destination.

copy!(client, source, destination, opts \\ [])

Like copy/4, but returns metadata directly and raises on failure.

delete(client, key, opts \\ [])

Deletes a Dropbox file or folder and returns its final metadata.

delete!(client, key, opts \\ [])

Like delete/3, but returns metadata directly and raises on failure.

delete_many(client, keys, opts \\ [])

Deletes several keys concurrently while preserving input order and isolating failures.

Accepts the same batch options as put_many/3; remaining options are passed to every deletion.

download(client, key, destination, opts \\ [])

@spec download(Magpie.Client.t(), binary(), Path.t(), keyword()) :: result(Path.t())

Streams key to a local destination and returns that destination.

The destination's parent must exist unless mkdir_p: true is passed. An existing destination is replaced only after Dropbox successfully sends the complete response. A :progress callback receives (transferred, total) as bytes arrive; pass the expected byte count as :size when it is known, otherwise total is nil.

download!(client, key, destination, opts \\ [])

Like download/4, but returns the destination directly and raises on failure.

exists?(client, key, opts \\ [])

@spec exists?(Magpie.Client.t(), binary(), keyword()) ::
  boolean() | {:error, Exception.t()}

Returns true, false for a missing key, or an error tuple for other failures.

get(client, key, opts \\ [])

@spec get(Magpie.Client.t(), binary(), keyword()) :: result(binary() | map())

Downloads key into memory and returns its bytes.

Pass with_headers: true to retain the %{body: body, headers: headers} response shape used by Magpie.Files.download/2.

get!(client, key, opts \\ [])

Like get/3, but returns the bytes directly and raises on failure.

list(client, prefix \\ "", opts \\ [])

@spec list(Magpie.Client.t(), binary(), keyword()) ::
  {:ok, [Magpie.Metadata.t()]} | {:error, Exception.t()}

Returns all entries below prefix, following every cursor page.

Unlike the lazy stream/3, this eager convenience returns Dropbox API and Req transport failures as {:error, exception}. This lets background jobs handle a failed listing without crashing the worker.

list!(client, prefix \\ "", opts \\ [])

Like list/3, but returns entries directly and raises on failure.

mkdir(client, key, opts \\ [])

Creates a Dropbox folder at key.

mkdir!(client, key, opts \\ [])

Like mkdir/3, but returns metadata directly and raises on failure.

move(client, source, destination, opts \\ [])

Moves a Dropbox file or folder to destination.

move!(client, source, destination, opts \\ [])

Like move/4, but returns metadata directly and raises on failure.

put(client, key, source, opts \\ [])

@spec put(Magpie.Client.t(), binary(), source(), keyword()) ::
  result(Magpie.FileMetadata.t()) | {:ok, :unchanged, Magpie.FileMetadata.t()}

Uploads a file, binary/iodata value, or stream to key.

Options are :mode, :if_rev, :autorename, :mute, :chunk_size, :session_threshold, :verify, :skip_unchanged and :progress. :if_rev performs a conditional update, :verify compares Dropbox's content hash after upload, and :skip_unchanged avoids uploading matching file/binary sources. Progress callbacks receive (transferred, total); total can be nil for streams.

put!(client, key, source, opts \\ [])

Like put/4, but raises on failure.

Returns metadata directly after an upload, or {:unchanged, metadata} when skip_unchanged: true finds identical remote content.

put_many(client, entries, opts \\ [])

Uploads several {key, source} or {key, source, options} entries concurrently.

Results keep input order and each item is isolated as {key, result}. Batch options are :max_concurrency, :timeout and a two-argument :on_progress callback receiving (key, result); remaining options are passed to every upload.

stat(client, key, opts \\ [])

Returns typed Dropbox metadata for key.

stat!(client, key, opts \\ [])

Like stat/3, but returns metadata directly and raises on failure.

stream(client, prefix \\ "", opts \\ [])

@spec stream(Magpie.Client.t(), binary(), keyword()) :: Enumerable.t()

Returns a lazy stream over every entry below prefix.

upload_url(client, key, opts \\ [])

@spec upload_url(Magpie.Client.t(), binary(), keyword()) ::
  {:ok, binary()} | {:error, Exception.t()}

Returns a one-use direct-upload URL for key.

upload_url!(client, key, opts \\ [])

Like upload_url/3, but returns the URL directly and raises on failure.

url(client, key, opts \\ [])

@spec url(Magpie.Client.t(), binary(), keyword()) ::
  {:ok, binary()} | {:error, Exception.t()}

Returns a temporary direct-download URL for key.

url!(client, key, opts \\ [])

Like url/3, but returns the URL directly and raises on failure.