livery_s3 (livery_s3 v0.1.0)
View SourceAn S3-compatible object storage client built on the livery client.
Build a client once with new/1 (endpoint, region, credentials, addressing
style), then call the operations: object CRUD with metadata and byte ranges,
bucket management, versioning, multipart upload, copy, batch delete, and
presigned URLs. Works against AWS S3 and S3-compatible stores (Garage, MinIO,
Ceph, …); use addressing => path (the default) for the widest compatibility.
C = livery_s3:new(#{
endpoint => <<"http://127.0.0.1:3900">>,
region => <<"garage">>,
access_key_id => <<"GK...">>,
secret_access_key => <<"...">>
}),
ok = livery_s3:create_bucket(C, <<"photos">>),
{ok, _} = livery_s3:put_object(C, <<"photos">>, <<"cat.jpg">>, Bytes,
#{content_type => <<"image/jpeg">>,
metadata => #{<<"album">> => <<"holiday">>}}),
{ok, #{body := Bytes}} = livery_s3:get_object(C, <<"photos">>, <<"cat.jpg">>).Transport, retries, and timeouts come from the underlying livery_client
layer stack; request signing is AWS Signature V4 (livery_s3_sigv4). All
operations return {ok, _}/ok or {error, Reason}; S3 error bodies surface
as {error, {s3, Code, Message, #{status => S, request_id => RId}}}.
Summary
Functions
Abort a multipart upload and discard its parts.
Complete a multipart upload. Parts is [{PartNumber, ETag}] in order (the
ETags returned by upload_part/6).
Server-side copy. See copy_object/6.
Server-side copy of SrcBucket/SrcKey to DstBucket/DstKey. Opts accepts the
same write options as put_object/5; supplying metadata switches the metadata
directive to REPLACE.
Create a bucket. See create_bucket/3.
Create a bucket. Opts: acl and, for AWS regions other than us-east-1,
location_constraint => Region (sent as a CreateBucketConfiguration). Most
S3-compatible stores need no body.
Begin a multipart upload, returning an upload id. See create_multipart_upload/4.
Begin a multipart upload. Opts accepts the put_object/5 write options.
Delete an (empty) bucket.
Delete an object. Opts may carry version_id to delete a version.
Delete up to 1000 objects in one request. Keys is a list of Key binaries or
{Key, VersionId} tuples. Result: #{deleted => [_], errors => [_]}.
Return the bucket's region. An empty LocationConstraint maps to us-east-1.
Read a bucket's versioning state: enabled, suspended, or none.
Download an object. See get_object/4.
Download an object. Opts
Check a bucket exists and is accessible. Missing bucket yields not_found.
Fetch object metadata without the body. Missing object yields not_found.
Fetch object metadata. Opts may carry version_id and the same conditional
headers as get_object/4 (304 -> not_modified, 412 ->
precondition_failed).
List all buckets owned by the caller.
List in-progress multipart uploads in a bucket. See list_multipart_uploads/3.
List in-progress multipart uploads. Opts: prefix, delimiter, max_uploads,
key_marker, upload_id_marker. Result: uploads
(#{key, upload_id, initiated}), common_prefixes, is_truncated,
next_key_marker, next_upload_id_marker.
List object versions. See list_object_versions/3.
List object versions and delete markers. Opts: prefix, delimiter,
max_keys, key_marker, version_id_marker. Result: versions,
delete_markers, is_truncated, next_key_marker, next_version_id_marker.
List objects (ListObjectsV2), one page. See list_objects/3.
List objects (ListObjectsV2). Opts: prefix, delimiter, max_keys,
continuation_token, start_after. Result: objects, common_prefixes,
is_truncated, next_continuation_token.
List every object, following continuation tokens. See list_objects/3.
List the parts uploaded so far for a multipart upload. See list_parts/5.
List uploaded parts. Opts: max_parts, part_number_marker. Result: parts
(#{part_number, etag, size, last_modified}), is_truncated,
next_part_number_marker.
Build a client. Options
Generate a presigned URL valid for Expires seconds. See presign/6.
Generate a presigned URL. Opts may carry version_id and the
response_content_* overrides (e.g. force a download filename). The URL signs
only the host header with an UNSIGNED-PAYLOAD, so it can be used directly by
a browser or any HTTP client.
Enable or suspend versioning on a bucket.
Upload an object. Body is iodata() or a {stream, Producer} body.
Upload an object with options: content_type, cache_control,
content_disposition, content_encoding, storage_class, acl, and
metadata (#{Name => Value} mapped to x-amz-meta-*).
Upload one part (1-based PartNumber). Returns #{etag => _}.
Upload a part by copying from an existing object. See upload_part_copy/8.
Upload a part by server-side copy from SrcBucket/SrcKey. Opts: range => {Start, End} copies a byte range of the source; version_id selects a source
version. Returns #{etag => _}.
Types
Functions
Abort a multipart upload and discard its parts.
-spec complete_multipart_upload(client(), bucket(), key(), binary(), [{pos_integer(), binary()}]) -> {ok, map()} | {error, reason()}.
Complete a multipart upload. Parts is [{PartNumber, ETag}] in order (the
ETags returned by upload_part/6).
Server-side copy. See copy_object/6.
-spec copy_object(client(), bucket(), key(), bucket(), key(), map()) -> {ok, map()} | {error, reason()}.
Server-side copy of SrcBucket/SrcKey to DstBucket/DstKey. Opts accepts the
same write options as put_object/5; supplying metadata switches the metadata
directive to REPLACE.
Create a bucket. See create_bucket/3.
Create a bucket. Opts: acl and, for AWS regions other than us-east-1,
location_constraint => Region (sent as a CreateBucketConfiguration). Most
S3-compatible stores need no body.
Begin a multipart upload, returning an upload id. See create_multipart_upload/4.
-spec create_multipart_upload(client(), bucket(), key(), map()) -> {ok, binary()} | {error, reason()}.
Begin a multipart upload. Opts accepts the put_object/5 write options.
Delete an (empty) bucket.
Delete an object. Opts may carry version_id to delete a version.
-spec delete_objects(client(), bucket(), [key() | {key(), binary()}]) -> {ok, map()} | {error, reason()}.
Delete up to 1000 objects in one request. Keys is a list of Key binaries or
{Key, VersionId} tuples. Result: #{deleted => [_], errors => [_]}.
Return the bucket's region. An empty LocationConstraint maps to us-east-1.
-spec get_bucket_versioning(client(), bucket()) -> {ok, enabled | suspended | none} | {error, reason()}.
Read a bucket's versioning state: enabled, suspended, or none.
Download an object. See get_object/4.
Download an object. Opts:
range-{Start, End},{Start, eof}, or{suffix, N}(last N bytes).version_id- read a specific version.stream- whentrue, the result'sbodyis a{stream, Reader}to drain withlivery_client:read/2; otherwise it is the full binary.- conditional headers
if_match,if_none_match,if_modified_since,if_unmodified_since; a304yields{error, not_modified}and a412yields{error, precondition_failed}. - response-header overrides
response_content_type,response_content_disposition,response_cache_control,response_content_encoding,response_content_language,response_expires.
The result map carries body, metadata, and (when present) content_type,
content_length, etag, last_modified, version_id.
Check a bucket exists and is accessible. Missing bucket yields not_found.
Fetch object metadata without the body. Missing object yields not_found.
Fetch object metadata. Opts may carry version_id and the same conditional
headers as get_object/4 (304 -> not_modified, 412 ->
precondition_failed).
List all buckets owned by the caller.
List in-progress multipart uploads in a bucket. See list_multipart_uploads/3.
List in-progress multipart uploads. Opts: prefix, delimiter, max_uploads,
key_marker, upload_id_marker. Result: uploads
(#{key, upload_id, initiated}), common_prefixes, is_truncated,
next_key_marker, next_upload_id_marker.
List object versions. See list_object_versions/3.
List object versions and delete markers. Opts: prefix, delimiter,
max_keys, key_marker, version_id_marker. Result: versions,
delete_markers, is_truncated, next_key_marker, next_version_id_marker.
List objects (ListObjectsV2), one page. See list_objects/3.
List objects (ListObjectsV2). Opts: prefix, delimiter, max_keys,
continuation_token, start_after. Result: objects, common_prefixes,
is_truncated, next_continuation_token.
List every object, following continuation tokens. See list_objects/3.
List the parts uploaded so far for a multipart upload. See list_parts/5.
List uploaded parts. Opts: max_parts, part_number_marker. Result: parts
(#{part_number, etag, size, last_modified}), is_truncated,
next_part_number_marker.
Build a client. Options:
endpoint(required) - base URL, e.g.<<"https://s3.eu-west-1.amazonaws.com">>or<<"http://127.0.0.1:3900">>.- Credentials (one required):
access_key_id+secret_access_key(+ optionalsession_token) for static keys, orcredentials => Providerto source them from the environment, the shared config file, EC2/ECS instance metadata, web-identity/STS, or thedefaultchain. Seelivery_s3_credentials. Refreshing providers (imds,{web_identity, _}) need thelivery_s3application started. region- default<<"us-east-1">>.addressing-path(default) orvirtual(bucket.host).timeout- per-request timeout in ms (default 30000), applied by the transport (hackneyrecv_timeout).
Resilience (built on livery_client layers, outermost to innermost
[concurrency, circuit_breaker, retry, balance, signing]):
retry-true(default),false, or an options map merged over the S3 defaults#{max => 3, backoff => {200, 2.0}, statuses => [429,500,502,503,504]}. Retries idempotent ops on transient statuses and connection errors, honoring aRetry-Afterheader when present; streamed request bodies and non-idempotent methods are never replayed.follow_region_redirects-true(default) follows S3 region redirects (301 PermanentRedirect/400 AuthorizationHeaderMalformed) by re-signing for the corrected region/host and retrying once;falsedisables it.circuit_breaker-true,false(default), or an options map (namedefaults to the endpoint authority). Trips on connection-level failures.concurrency- a positive integer cap on in-flight requests, orfalse(default); over the cap returns{error, overloaded}.endpoints- a list of base URLs to spread/fail over (path-style only, same region and credentials), or passbalance => Mapfor full control.
circuit_breaker and endpoints/balance are ETS-backed and require the
livery application to be started. retry and concurrency need nothing.
stack- a fulllivery_clientlayer stack that bypasses the builders above (signing is still appended innermost). The spawn-basedlivery_client:timeout/1layer is incompatible with streamed downloads, so it is never added by default.adapter,adapter_opts- forwarded tolivery_client:new/1.
-spec presign(client(), atom() | binary(), bucket(), key(), pos_integer()) -> {ok, binary()} | {error, reason()}.
Generate a presigned URL valid for Expires seconds. See presign/6.
-spec presign(client(), atom() | binary(), bucket(), key(), pos_integer(), map()) -> {ok, binary()} | {error, reason()}.
Generate a presigned URL. Opts may carry version_id and the
response_content_* overrides (e.g. force a download filename). The URL signs
only the host header with an UNSIGNED-PAYLOAD, so it can be used directly by
a browser or any HTTP client.
Enable or suspend versioning on a bucket.
Upload an object. Body is iodata() or a {stream, Producer} body.
Upload an object with options: content_type, cache_control,
content_disposition, content_encoding, storage_class, acl, and
metadata (#{Name => Value} mapped to x-amz-meta-*).
Conditional writes (backend-dependent): if_none_match => <<"*">> puts only if
the object does not exist, if_match => ETag only if it matches; a failed
precondition yields {error, precondition_failed}. content_md5 => true adds a
base64 Content-MD5 integrity header (full-body uploads only).
-spec upload_part(client(), bucket(), key(), binary(), pos_integer(), body()) -> {ok, map()} | {error, reason()}.
Upload one part (1-based PartNumber). Returns #{etag => _}.
-spec upload_part_copy(client(), bucket(), key(), binary(), pos_integer(), bucket(), key()) -> {ok, map()} | {error, reason()}.
Upload a part by copying from an existing object. See upload_part_copy/8.
-spec upload_part_copy(client(), bucket(), key(), binary(), pos_integer(), bucket(), key(), map()) -> {ok, map()} | {error, reason()}.
Upload a part by server-side copy from SrcBucket/SrcKey. Opts: range => {Start, End} copies a byte range of the source; version_id selects a source
version. Returns #{etag => _}.