Bloccs.Lineage (bloccs v0.10.0)

Copy Markdown View Source

Runtime internals

Infrastructure called by compiler-generated pipelines — not part of the stable user API. You drive this through manifests, not by calling it directly; signatures may change between minor versions.

Per-message causal lineage — the identity that lets a single logical message be tracked across the hops it takes through a network.

Each message carries a small lineage map in its Broadway.Message metadata (under key/0), never in the payload — payloads have a declared Name@N schema we must not pollute:

%{msg_id: id, parents: [id], trace_id: id}
  • msg_id — a unique id for this message instance, minted at each emit.
  • parents — the msg_ids of the input message(s) that caused this emit. One parent for a transform/split, many for a fan-in (batch, join).
  • trace_id — a root-correlation convenience: the ingress msg_id, propagated unchanged on 1:1 and fan-out, and minted fresh on fan-in (a merge is a new logical message; parents preserves the cross-trace links).

The primitive is msg_id + parents — a causal DAG. trace_id is a derived convenience for the common 1:1 chain and for cheap grouping.

Ids are System.unique_integer([:positive, :monotonic]) — cheap on the hot path and monotonic within a runtime (single-node assumption for v1).

Summary

Types

ctx()
id()
t()

Functions

child(arg1)

Lineage context for a 1:1 / split / filter emit from a single parent: the child inherits the parent's trace_id and lists it as its sole parent. A nil parent (an orphan emit, e.g. a direct Router.dispatch/4) starts a fresh trace with no parents.

ensure(meta)

Ensure a metadata map carries a lineage — inject a root/0 if it has none. Called when a producer builds a message, so external ingress (a bare push with no lineage) becomes a tracked root while internal hops keep theirs.

key()

The Broadway.Message metadata key lineage is carried under.

merge(parents)

Lineage context for a fan-in (batch / join) emit from many parents: a fresh trace_id (the merge is a new logical message), with every input listed as a parent. nil / lineage-less inputs are ignored.

new_id()

Mint a fresh, unique, monotonic id.

of(meta)

Extract the lineage carried in a message's metadata, or nil if absent.

root()

A root lineage for a message with no upstream (external ingress).

stamp(map)

Mint a full lineage for an emitted message from a context, generating its msg_id.

Types

ctx()

@type ctx() :: %{parents: [id()], trace_id: id()}

id()

@type id() :: pos_integer()

t()

@type t() :: %{msg_id: id(), parents: [id()], trace_id: id()}

Functions

child(arg1)

@spec child(t() | nil) :: ctx()

Lineage context for a 1:1 / split / filter emit from a single parent: the child inherits the parent's trace_id and lists it as its sole parent. A nil parent (an orphan emit, e.g. a direct Router.dispatch/4) starts a fresh trace with no parents.

ensure(meta)

@spec ensure(map()) :: map()

Ensure a metadata map carries a lineage — inject a root/0 if it has none. Called when a producer builds a message, so external ingress (a bare push with no lineage) becomes a tracked root while internal hops keep theirs.

key()

@spec key() :: :bloccs_lineage

The Broadway.Message metadata key lineage is carried under.

merge(parents)

@spec merge([t() | nil]) :: ctx()

Lineage context for a fan-in (batch / join) emit from many parents: a fresh trace_id (the merge is a new logical message), with every input listed as a parent. nil / lineage-less inputs are ignored.

new_id()

@spec new_id() :: id()

Mint a fresh, unique, monotonic id.

of(meta)

@spec of(map() | term()) :: t() | nil

Extract the lineage carried in a message's metadata, or nil if absent.

root()

@spec root() :: %{msg_id: id(), parents: [], trace_id: id()}

A root lineage for a message with no upstream (external ingress).

stamp(map)

@spec stamp(ctx()) :: t()

Mint a full lineage for an emitted message from a context, generating its msg_id.