macula_group_keys (macula v17.1.1)

View Source

A sealed group's distributor (docs/design/DESIGN_E2E_SEALED_PUBSUB.md §3, §5): the handler of <org>/group_keys_v1 and the epochs of every group it serves.

The application advertises the handler as any provider advertises one (its mcl_om capability, or macula_response:advertise_direct/7 kept alive with reuse_sup), with advertise_opts/0 merged into its options: confidential => required, so the provider's link answers a clear call sealed_required and a key never travels clear, AND auth => handler_authorizes, so the link hands the handler the token and proofs the caller signed into the request, and the handler verifies them. The check cannot sit in an advertise gate: the group a pull names travels inside the sealed payload, where no gate sees it (macula#129). With the node's kem_advertise switched off those options refuse to advertise at all (kem_advertise_disabled), rather than advertise a keyless distributor. The member side refuses a keyless distributor too (macula_group_keyring pulls with confidential => required): each side enforces it on its own.

Each group has one access policy, chosen at start (groups):

  • org_grant, the default for a group not listed: the org's group_keys grant, {realm_member_required, OrgKeyId, <<"group_keys">>} on <org>/group_keys_v1, the check the advertise gate made before;
  • {realm_capability, RealmKeyId, Mode}: the realm key's pubsub/subscribe, issued to the caller, covering the whole group, its prefix mri:topic:P and everything below it mri:topic:P/* (macula_ucan:authorize/3 with a group). A role the realm grants on P/* alone does not admit: a group key opens P too. Mode is refusing, which admits on the realm capability alone, or counting, the default, which counts a pull the realm capability would refuse and admits it when the same token passes the org grant. Counting is therefore fail-open by design, the union of two policies, and must show as counting, never as protected, until its would-be refusals read zero. A realm_capability group needs the distributor's seal policy required: its events must never travel clear.

Each listed prefix is its own group: listing P restricts the group P, not a group P/sub pulled under its own prefix, which needs its own entry. Either check verifies the token's audience against the caller, its validity window at the time of the pull (a past epoch's pull too), and its chain of proofs up to the key the policy names. A refused pull answers not_authorized: <can> on <resource>, naming what it lacked. Every pull of a realm_capability group is counted per realm, group and outcome (admitted, would_refuse, refused), read by counts/1, and a refusal or would-be refusal is logged at most once a minute per group and outcome.

The handler then decides from the caller's wire-authenticated node id: a node in the application's removed set is refused every epoch, and so is a node that is not a live realm member, read from the realm's slot for its endorsement (macula_record:realm_member_endorsement_key/2 and macula_hyparview_endorsement:slot_endorsement/4, which answers withdrawn for the realm's tombstone). Membership is checked at the time of the call, for a past epoch too: a subscriber catching up on one it missed is still a member, and one that is not has no business reading the group. Each lookup runs in a process of its own, so a slow one holds up only the pull that waits on it: every other pull, of any group, is answered meanwhile. A caller has one lookup at a time: its further pulls, up to 16, wait on the one in flight and share its answer, and the removed set is asked again when the answer comes. A lookup gets lookup_wait_ms (default 10 s), and at most max_lookups (default 64) callers are looked up at once; a pull past any of these is membership_unknown, so its member retries. A lookup that fails in any other way, a pool given as a pid that has died included, is membership_unknown too, and the distributor stays up.

A call names the group (prefix) and the epoch it wants: current, or a past one by id. The reply carries the group's policy and the epochs: the current one, and the next one too inside the current's ahead window (macula_group_epoch). Epoch keys are kept in memory only. A refusal is {error, Reason}, which macula answers as a sealed provider error with code handler_error and the reason as its detail: not_authorized: ... (above), not_a_member, membership_unknown (the lookup failed; retry), unknown_epoch, epoch_expired, unknown_group (a prefix this distributor's org does not own), too_many_groups (below) or malformed_pull (a payload that is not a map).

A group not listed is served on its first pull, and kept as a row of the table. At most max_groups (default 1024) unlisted groups are kept: past that, the rows whose epochs are all past their keeping are dropped (a listed group's too, which starts afresh on its next pull), and if none is, the current epoch of a new unlisted group is refused too_many_groups. A listed group, and a group already kept, is always served, so a caller asking for prefix after prefix cannot grow the table past the bound or push out a group its members use.

The epochs live in a private table the distributor's process owns, and its state holds only that table's id: no other process can read the table, and neither sys:get_state/1, sys:get_status/1 nor a crash report shows a key. format_status/1 redacts every epoch's key as a second line of defence. A crash dump lists a table's size, never its rows; it does copy every process heap, where a reply passes on its way out, so a node holding a group's keys runs with crash dumps off.

Summary

Functions

The options the distributor's procedure is advertised with, merged into the application's own: the call's own token handed to the handler, which checks it per group, and sealed calls only.

The pulls of every realm_capability group so far, by realm, group and outcome.

The <org>/group_keys_v1 handler for a distributor, to advertise. It waits for the distributor longer than a membership lookup may take (10 s) and less than a member's pull deadline (15 s), so a slow lookup answers membership_unknown, and no answer is made that no member still waits for.

Start a distributor for org's groups in realm (the realm id), checking tokens in profile. org_key_id names the org key whose group_keys grant an org_grant group admits; groups maps a group's prefix to its access policy, {realm_capability, RealmKeyId} meaning counting mode. A listed prefix the org does not own is refused as {error, {unknown_group, Prefix}}, an access of another shape as {error, {invalid_group_access, Prefix}}, and a realm_capability group under a seal policy other than required as {error, {restricted_group_not_required, Prefix}}: nothing is started. membership defaults to the realm slot read over pool, which then needs realm_key_id too; removed defaults to nobody removed. Name the pool: pool as its registered name is looked up at every read, so a pool its supervisor restarts is found again, and while no pool holds the name a read is membership_unknown. A pid is read as given, and once that pool has died every read is membership_unknown. max_groups, max_lookups and lookup_wait_ms bound what pulls can cost (above). The application keeps its removed set across restarts: a set lost to a restart re-admits every removed member whose grant still verifies.

Types

access/0

-type access() :: org_grant | {realm_capability, <<_:256>>} | {realm_capability, <<_:256>>, mode()}.

mode/0

-type mode() :: counting | refusing.

options/0

-type options() ::
          #{org := binary(),
            policy := policy(),
            org_key_id := <<_:256>>,
            realm := <<_:256>>,
            profile := macula_crypto_profile:profile(),
            groups => #{binary() => access()},
            rotate_after_ms => pos_integer(),
            now => fun(() -> integer()),
            membership => fun((<<_:256>>) -> ok | {error, not_a_member | membership_unknown}),
            removed => fun((<<_:256>>) -> boolean()),
            pool => pid() | atom(),
            realm_key_id => <<_:256>>,
            max_groups => pos_integer(),
            max_lookups => pos_integer(),
            lookup_wait_ms => pos_integer()}.

outcome/0

-type outcome() :: admitted | would_refuse | refused.

policy/0

-type policy() :: required | preferred | off.

Functions

counts(Pid)

-spec counts(pid()) -> #{{<<_:256>>, binary(), outcome()} => pos_integer()}.

The pulls of every realm_capability group so far, by realm, group and outcome.

handler(Pid)

-spec handler(pid()) -> fun((term()) -> map() | {error, atom() | binary()}).

The <org>/group_keys_v1 handler for a distributor, to advertise. It waits for the distributor longer than a membership lookup may take (10 s) and less than a member's pull deadline (15 s), so a slow lookup answers membership_unknown, and no answer is made that no member still waits for.

start_link(Opts)

-spec start_link(options()) -> {ok, pid()} | {error, term()}.

Start a distributor for org's groups in realm (the realm id), checking tokens in profile. org_key_id names the org key whose group_keys grant an org_grant group admits; groups maps a group's prefix to its access policy, {realm_capability, RealmKeyId} meaning counting mode. A listed prefix the org does not own is refused as {error, {unknown_group, Prefix}}, an access of another shape as {error, {invalid_group_access, Prefix}}, and a realm_capability group under a seal policy other than required as {error, {restricted_group_not_required, Prefix}}: nothing is started. membership defaults to the realm slot read over pool, which then needs realm_key_id too; removed defaults to nobody removed. Name the pool: pool as its registered name is looked up at every read, so a pool its supervisor restarts is found again, and while no pool holds the name a read is membership_unknown. A pid is read as given, and once that pool has died every read is membership_unknown. max_groups, max_lookups and lookup_wait_ms bound what pulls can cost (above). The application keeps its removed set across restarts: a set lost to a restart re-admits every removed member whose grant still verifies.