ImagePipe.Source.S3 (image_pipe v0.1.0)

Source adapter for objects in S3 and S3-compatible storage.

media: [
  adapter: ImagePipe.Source.S3,
  match: [scheme: "s3"],
  options: [
    default: [
      region: "us-east-1",
      endpoint: "https://s3.us-east-1.amazonaws.com",
      credentials: {:provider, ImagePipe.Source.S3.InstanceRole, []}
    ]
  ]
]

It resolves object sources written s3://bucket/key or s3://bucket/key?revision. Setting up a mount is covered in Serving images from S3.

A revision is an S3 version ID, written as the whole query (?3HL4kqtJlcpX, not ?versionId=3HL4kqtJlcpX). The adapter requests that version, and the object is treated as immutable. The response must carry a matching x-amz-version-id header, or the fetch fails with 502. Without a revision, the object's Cache-Control, ETag, and Last-Modified headers set its cache lifetime, as for an HTTP source.

Options

• :default (keyword/0) - Bucket settings for every bucket. Must include :region and :endpoint. The default value is [].

• :buckets (%{String.t() => keyword()}) - Bucket settings per bucket name, each merged over :default. When given, only the listed buckets are served, and other buckets are not found. Omitted, every bucket is served.

Bucket settings

:default and each entry of :buckets take these settings:

• :region (String.t/0) - Required. Region used to sign requests, such as "us-east-1".

• :endpoint (String.t/0) - Required. Base URL of the S3 API, such as "https://s3.us-east-1.amazonaws.com", with no path, query, or fragment. Objects are requested path-style, as <endpoint>/<bucket>/<key>.

• :credentials ({:static, keyword()} or {:provider, module(), keyword()}) - Credentials that sign requests. Every bucket needs them, from :default or its own settings. See Credentials.

• :req_options (keyword/0) - Options for Req.request/1, such as headers:. The adapter drops :url, :base_url, :method, :body, :params, :into, :retry, :redirect, :max_redirects, :auth, and :aws_sigv4, and the authorization, host, x-amz-content-sha256, and x-amz-security-token headers. It also drops range, accept, and accept-encoding headers, except on a source with internal_cache: :disabled that isn't immutable. Requests for one object must always return the same bytes. The default value is [].

• :receive_timeout (non_neg_integer/0) - Milliseconds to wait for the response and between body chunks. The default value is 5000.

• :fetch_timeout (pos_integer/0) - Milliseconds for the whole download, from the request to the last body byte. A connection attempt can run past it by up to connect_timeout. Requests waiting for the same object give up after waiting this long plus one second. The default value is 30000.

• :connect_timeout (non_neg_integer/0) - Milliseconds to wait for a connection. The default value is 5000.

• :pool_timeout (non_neg_integer/0) - Milliseconds to wait for a free connection from the pool. The default value is 5000.

• :stable (:auto or :immutable) - :immutable marks an immutable source: an identifier always names the same bytes, so its originals and processed images never expire and are never revalidated. With :auto, each original is identified by its content. See immutable sources. The default value is :auto.

• :cache_policy (keyword/0) - Cache policy for this source, with the fields of ImagePipe.Source.CachePolicy. Each field it sets replaces that field of the :source_cache_policy of ImagePipe.config/1. A source with stable: :immutable can't set a :freshness duration or a :stale_while_revalidate window. The default value is [].

• :internal_cache (:auto, :enabled, or :disabled) - :disabled keeps this source's originals and processed images out of the caches, including images that use this source for a watermark. :auto and :enabled cache what the source's storage policy allows. The default value is :auto.

• :http_cache (:inherit, :validators, :auto, :public, or :private) - HTTP cache headers for responses from this source. :inherit uses the mount's :http_cache (see ImagePipe.Plug), and any other value replaces it. See per-source modes. The default value is :inherit.

Credentials

• {:static, access_key_id: id, secret_access_key: secret} - fixed keys. Add token: for temporary credentials with a session token. A nil token is the same as leaving token: out, and an empty string is rejected.
• {:provider, module, options} - a module implementing ImagePipe.Source.S3.CredentialProvider, which fetches credentials and refreshes them before they expire. ImagePipe includes ImagePipe.Source.S3.InstanceRole, ImagePipe.Source.S3.ContainerCredentials, ImagePipe.Source.S3.WebIdentity, and ImagePipe.Source.S3.AssumeRole.

Credentials from a provider are cached per provider, options, and bucket. Expired credentials are never sent. When they can't be refreshed, requests fail with {:source, :credentials_unavailable}, which answers 500.