ReqS3 (req_s3 v0.2.0)
Req
plugin for Amazon S3 and S3 API-compatible services.
ReqS3 handles a custom s3://
url scheme that supports two endpoints:
s3://<bucket> # list bucket items
s3://<bucket>/<item> # get item content
Examples
iex> req = Req.new() |> ReqS3.attach()
iex> Req.get!(req, url: "s3://ossci-datasets").body
%{
"ListBucketResult" => %{
"Contents" => [
%{"Key" => "mnist/", ...},
%{"Key" => "mnist/t10k-images-idx3-ubyte.gz", ...},
...
],
"Name" => "ossci-datasets",
...
}
}
iex> req = Req.new() |> ReqS3.attach()
iex> body = Req.get!(req, url: "s3://ossci-datasets/mnist/t10k-images-idx3-ubyte.gz").body
iex> <<_::32, n_images::32, n_rows::32, n_cols::32, _body::binary>> = body
iex> {n_images, n_rows, n_cols}
{10_000, 28, 28}
Pre-signing
See presign_url/1
and presign_form/1
.
Summary
Functions
Attaches the plugin.
Returns presigned form for upload.
Returns a presigned URL for fetching bucket object contents.
Functions
attach(request, options \\ [])
Attaches the plugin.
presign_form(options)
Returns presigned form for upload.
Options
:access_key_id
- the access key id.:secret_access_key
- the secret access key.:region
- the S3 region, defaults to"us-east-1"
.:bucket
- the S3 bucket.:key
- the S3 bucket key.:content_type
- if set, the content-type of the uploaded object.:max_size
- if set, the maximum size of the uploaded object.:expires_in
- the time in milliseconds before the signed upload expires. Defaults to 1h (60 * 60 * 1000
milliseconds).:datetime
- the request datetime, defaults toDateTime.utc_now(:second)
.
Examples
iex> options = [
...> access_key_id: System.fetch_env!("AWS_ACCESS_KEY_ID"),
...> secret_access_key: System.fetch_env!("AWS_SECRET_ACCESS_KEY"),
...> bucket: "bucket1",
...> key: "key1",
...> expires_in: :timer.hours(1)
...> ]
iex> %{url: url, fields: fields} = ReqS3.presign_form(options)
iex> url
"https://bucket1.s3.amazonaws.com"
iex> fields
%{
"key" => "key1",
"policy" => "eyJjb25kaXRpb25z...ifQ==",
"x-amz-algorithm" => "AWS4-HMAC-SHA256",
"x-amz-credential" => "AKIA.../20240528/us-east-1/s3/aws4_request",
"x-amz-date" => "20240528T105226Z",
"x-amz-server-side-encryption" => "AES256",
"x-amz-signature" => "465315d202fbb2ce081f79fca755a958a18ff68d253e6d2a611ca4b2292d8925"
}
presign_url(options)
Returns a presigned URL for fetching bucket object contents.
Options
:access_key_id
- the AWS access key id.:secret_access_key
- the AWS secret access key.:region
- if set, AWS region. Defaults to"us-east-1"
.:url
- the URL to presign, for example:"https://bucket.s3.amazonaws.com"
ors3://bucket
.Instead of passing the
:url
option, you can also pass:bucket
and:key
options which will generate ahttps://{bucket}.s3.amazonaws.com/{key}
url.
Examples
iex> options = [
...> access_key_id: System.fetch_env!("AWS_ACCESS_KEY_ID"),
...> secret_access_key: System.fetch_env!("AWS_SECRET_ACCESS_KEY"),
...> ]
iex> req = Req.new() |> ReqS3.attach(aws_sigv4: options)
iex> %{status: 200} = Req.put!(req, url: "s3://wojtekmach-test/key1", body: "Hello, World!")
iex> url = ReqS3.presign_url([url: "s3://wojtekmach-test/key1"] ++ options)
iex> "https://wojtekmach-test.s3.amazonaws.com/key1?X-Amz-Algorithm=AWS4-HMAC" <> _ = url
iex> %{status: 200, body: body} = Req.get!(url)
iex> body
"Hello, World!"