Roux.Blob.Trace.Pack (roux v0.3.2)

Copy Markdown View Source

Batches small verifying traces into immutable, indexed packs.

with_group/4 selects a stable storage group for fetch/3 and put/5 in the calling process. Use the same group in later sessions; other groups are not searched. Loose traces remain readable as a migration path. Calls outside a group use the ordinary Roux.Blob.Trace store.

Each record is compressed and hashed separately. One CAS blob holds their concatenated bytes; a small index maps name hashes to offsets, versions and observation hashes. The index also lists referenced blobs so the existing collector can retain them without decoding every record.

Writes flush at the configured size bound and before a successful return. Looking up a pending name flushes first, providing read-your-writes. A killed process loses only its unpublished batch; readers never see partial packs. Worker processes do not inherit a group. Nested groups restore their caller's group on return.

write: :loose reads the group's packs but writes ordinary traces. Use it for isolated lookups that cannot amortize a batch. A nested call to the active group keeps that group's write policy and size limits.

lookup: :snapshot discovers packed indexes once per group scope, refreshing after this process publishes a pack. Concurrent publications can be missed until the next scope. Use it for pure computations whose traces are still checked against their observations. The default :latest discovers packs on every lookup. Loose traces are always read afresh.

Collection operates on whole packs. Using one record retains its neighbors; keep: bounds lookup history outside the refresh window, but reclaiming individual obsolete records requires repacking. Choose bounded groups, such as one source module, rather than an entire project.

Summary

Functions

Reads the newest matching versions from the current group and loose storage.

Keeps a trace in the current group, or writes a loose trace outside one.

Runs fun with bounded packed writes in group, returning its result.

Functions

fetch(store, name, opts \\ [])

@spec fetch(Roux.Blob.t(), term(), keyword()) :: [Roux.Blob.Trace.t()]

Reads the newest matching versions from the current group and loose storage.

put(store, name, deps, value, opts \\ [])

@spec put(Roux.Blob.t(), term(), [Roux.Blob.Trace.dep()], term(), keyword()) ::
  :ok | {:error, File.posix()}

Keeps a trace in the current group, or writes a loose trace outside one.

with_group(store, group, fun, opts \\ [])

@spec with_group(Roux.Blob.t(), term(), (-> result), keyword()) :: result
when result: var

Runs fun with bounded packed writes in group, returning its result.