GcpGcs.Download (GcpGcs v0.2.0)

Copy Markdown View Source

Downloads object contents from Google Cloud Storage.

All downloads use the JSON API media endpoint (?alt=media). download/3 buffers the whole object in memory; download_to_file/4 and stream/5 stream the body in chunks for constant-memory transfers of large objects.

Exposed through the top-level GcpGcs module.

Summary

Functions

Downloads an object and returns its full contents as a binary.

Streams an object's contents to a local file at path, using constant memory.

Streams an object's contents through reducer, folding from acc.

Functions

download(bucket, object, opts \\ [])

@spec download(String.t(), String.t(), keyword()) ::
  {:ok, binary()} | {:error, GcpGcs.Error.t()}

Downloads an object and returns its full contents as a binary.

Prefer download_to_file/4 or stream/5 for large objects.

Options

  • :generation — a specific object generation
  • :timeout — request timeout in ms

download_to_file(bucket, object, path, opts \\ [])

@spec download_to_file(String.t(), String.t(), Path.t(), keyword()) ::
  :ok | {:error, GcpGcs.Error.t()}

Streams an object's contents to a local file at path, using constant memory.

Returns :ok. On error the partially-written file is removed.

stream(bucket, object, opts, acc, reducer)

@spec stream(String.t(), String.t(), keyword(), acc, (term(), acc -> acc)) ::
  {:ok, acc} | {:error, GcpGcs.Error.t()}
when acc: term()

Streams an object's contents through reducer, folding from acc.

reducer receives {:headers, headers} once, then {:data, binary} for each body chunk, returning the updated accumulator. Returns {:ok, acc}.

Example

# Count bytes without buffering the object
{:ok, total} =
  GcpGcs.Download.stream("my-bucket", "big.bin", [], 0, fn
    {:data, chunk}, n -> n + byte_size(chunk)
    _other, n -> n
  end)