ExStorageService.Storage.Engine (ex_storage_service v0.6.3)

Copy Markdown View Source

Storage engine facade over the global content-addressable store (ExStorageService.Storage.CAS).

PUTs stream data to a CAS tmp file — computing SHA-256 and MD5 in a single pass in the calling process — then commit with an atomic rename. GETs resolve packed blobs first, then fall back to loose global CAS and legacy bucket-local paths ({data_root}/{bucket}/objects/...) for content written before the global-CAS migration (see ExStorageService.Storage.Migration).

The GenServer exists only to create the storage directories at boot; every read/write operation is a plain function.

Summary

Functions

Returns a specification to start this module under a supervisor.

Returns the data_root path configured for the engine.

Ensure the bucket directory structure exists (legacy layout; still used for multipart part staging).

Returns the file path for the given content hash, suitable for sendfile.

Resolve a blob to a servable location: a whole file (loose CAS or legacy layout) or a slice of a pack file. Serving code uses the offset/length forms of send_file for pack slices, preserving zero-copy and Range.

Construct the legacy bucket-local filesystem path for a content hash. Only pre-migration content lives here; new writes go to the CAS.

Ensures the given content hash is present in the global CAS, promoting it from the legacy bucket-local layout when necessary (used by metadata-only CopyObject and by the migration task).

Store object data in the global CAS, computing SHA-256 and MD5 in a single pass. data_or_stream can be a binary or an Enumerable of binary chunks; streams are enumerated in the calling process, so this is safe for Plug.Conn.read_body/2-backed streams.

Stream-aware PUT. Kept as a separate name for existing callers; the write always happens in the calling process now, so this is equivalent to put_object/5.

Read a blob's bytes regardless of physical location.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

data_root()

Returns the data_root path configured for the engine.

ensure_bucket_dirs(bucket)

Ensure the bucket directory structure exists (legacy layout; still used for multipart part staging).

get_object(bucket, content_hash)

Returns the file path for the given content hash, suitable for sendfile.

Resolves the global CAS first, then falls back to the legacy bucket-local path for content that predates the CAS migration.

get_object_location(bucket, content_hash)

Resolve a blob to a servable location: a whole file (loose CAS or legacy layout) or a slice of a pack file. Serving code uses the offset/length forms of send_file for pack slices, preserving zero-copy and Range.

legacy_content_path(data_root, bucket, content_hash)

Construct the legacy bucket-local filesystem path for a content hash. Only pre-migration content lives here; new writes go to the CAS.

promote_to_global(bucket, content_hash)

Ensures the given content hash is present in the global CAS, promoting it from the legacy bucket-local layout when necessary (used by metadata-only CopyObject and by the migration task).

Reads of the legacy source keep working after promotion because get_object/2 checks the CAS first.

put_object(bucket, key, data_or_stream, content_type \\ "application/octet-stream", metadata \\ %{})

Store object data in the global CAS, computing SHA-256 and MD5 in a single pass. data_or_stream can be a binary or an Enumerable of binary chunks; streams are enumerated in the calling process, so this is safe for Plug.Conn.read_body/2-backed streams.

Returns {:ok, {content_hash, etag, size}} on success.

put_object_stream(bucket, key, stream, content_type \\ "application/octet-stream", metadata \\ %{})

Stream-aware PUT. Kept as a separate name for existing callers; the write always happens in the calling process now, so this is equivalent to put_object/5.

read_object(bucket, content_hash)

Read a blob's bytes regardless of physical location.

start_link(opts)