Rindle.Storage.S3 (Rindle v0.3.1)

Copy Markdown View Source

S3-compatible storage adapter powered by ExAws.

tus single-node constraint

The S3 tus backing (upload_part_stream/5) buffers each PATCH's sub-5-MiB tail remainder on node-local disk under Rindle.tmp/tus/, while the authoritative cross-PATCH bookkeeping (offset, upload_id, committed parts) lives in the shared DB. Because the tail buffer is node-local, the S3 tus backing REQUIRES single-node or sticky-session routing: a resumed PATCH must reach the same node that holds the in-progress tail buffer.

A cross-node resume — where the DB implies buffered bytes (a non-empty upload_id together with EITHER at least one committed part OR a persisted offset greater than length(parts) * @s3_min_part_size) but the expected tail file is absent on this node — is detected and fails loudly with {:error, :tus_tail_missing} rather than silently re-slicing from a fresh empty tail (which would corrupt the assembled object). This covers the pre-first-part window too: a first PATCH under 5 MiB buffers a node-local tail without committing any part (parts: [], offset > 0), so a misrouted resume in that window also fails loudly instead of dropping the buffered bytes. A brand-new FIRST PATCH (offset == 0) is never falsely guarded. Multi-node operators MUST pin tus PATCHes to a single node (sticky sessions) or accept this loud failure on misrouted resumes.

Public (browser-facing) delivery endpoint

Browser-facing presigned URLs (url/2, presigned_put/3, presigned_upload_part/5) are signed using the :ex_aws, :s3 config by default. When the cluster-internal S3 endpoint differs from the endpoint the browser can reach — split-horizon DNS, a public/CDN/edge host, or a dev Docker setup where the server talks to minio:9000 in-network but the browser must use a published localhost:<port> — configure a :public_endpoint for this adapter:

config :rindle, Rindle.Storage.S3,
  bucket: "my-bucket",
  public_endpoint: [scheme: "https://", host: "cdn.example.com", port: 443]

Only :scheme, :host, and :port are read, and they apply ONLY to presigned URL signing — server-side store/download/head/multipart ops keep using the :ex_aws, :s3 endpoint. Because the S3 signature binds the host header (SignedHeaders=host), the configured public host MUST be exactly the host the browser requests. Leave it unset for identical pre-existing behaviour.

Summary

Functions

Canonical reaper-facing path of the on-disk tail buffer for a tus session.

Functions

tus_tail_path(session_id, opts \\ [])

@spec tus_tail_path(
  binary(),
  keyword()
) :: Path.t()

Canonical reaper-facing path of the on-disk tail buffer for a tus session.

Returns the EXACT file upload_part_stream/5 writes its sub-5-MiB tail remainder to for session_id, so cleanup code (the orphan reaper / Rindle.Ops.UploadMaintenance) can delete the real file rather than guessing at the encoding. The adapter owns the one canonical tail-path computation here: the path is Base.url_encode64(session_id, padding: false) <> ".tail" under the sweepable Rindle.tmp/tus/ root — never the raw id (CR-02).

Pass :root to override the base dir (used by tests for per-test isolation). Delegates to the private tail_path/2, threading session_id as both the key and the :session_id opt so the encoding is identical regardless of how the original PATCH was keyed. There is exactly one Base.url_encode64 site (tail_filename/1); this helper does not re-encode.