All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
Unreleased
[0.2.0] - 2026-09-16
Operational release: a projection can now say how far behind it is, be
tested without a pipeline, be inspected after it dead-letters, be rebuilt,
and survive a rolling deploy. Upgrading from 0.1.x means one migration —
Scriba.Migrations.up(from: 1) — and nothing else breaks.
Changed
Documented the assumption
:positioncarries: a global, monotonic, numeric ordinal over every event a projection consumes. Dedup, cursor monotonicity, the watermark and:start_fromall rest on it, and Commanded'sevent_numbersatisfies it — which is also why the cursor carries over fromcommanded_ecto_projections. A store whose position is not a single increasing integer (commit/prepare pairs, a vector clock) is a design question rather than an adapter detail, and is left open rather than guessed at.bench/gained a suite of experiments that need a real event store rather than a real database: acknowledgement loss under a straggler, standby takeover, the watermark end to end, subscription contention, and thebroadway_dashboardintegration. Each one refutes or confirms a claim the documentation makes, which is why they are kept rather than deleted once the question was answered. The contention experiment is the reason standby exists: it disproved a predicted crash loop and found the real gap, which was that nothing retried.Documented that the target is the transaction boundary — read-model rows, cursor advances and dead letters commit together or not at all — and that cursor and dead-letter storage will not be split into a separate behaviour. Splitting them would either leave them in the same transaction anyway or turn effectively-once delivery into at-least-once with a crash window.
Other sources and targets are not being prepared for speculatively. An interface with one implementation behind it encodes that implementation's assumptions, which is how the acknowledgement defect fixed in 0.1.3 survived a green test suite. The README and
Scriba.Targetnow invite an issue instead: another adapter gets built with whoever needs it.A producer that cannot get its subscription now stands by instead of failing. A persistent subscription admits one subscriber, so on a multi-node deployment every node but one is refused. Previously the loser raised during
init, which meantDynamicSupervisornever adopted it andstart_projection/1returned an error — no crash loop, but no standby either: nothing retried, so a failover had nobody to fail over to.The producer now starts, retries in the background, and acquires the subscription when the holder releases it. The retry curve has three phases, because two different failures share the path: five attempts in milliseconds for the reap race after a deliberate producer death, then one a second for half a minute — a killed tree holds its names until its slowest in-flight handler returns — then about once a minute with jitter, so standbys that started together do not retry in lockstep. Configuration errors still raise; retrying those forever would hide them.
Two new telemetry events make a takeover observable:
[:scriba, :source, :standby]per failed attempt, and[:scriba, :source, :subscribed]when acquired. A standby's projection reports:running, because its pipeline is up — the telemetry is what distinguishes the node doing the work.This also covers a cutover from
commanded_ecto_projections: Scriba can be started while the old projector still holds the subscription name, and picks up when it stops.
Added
REBUILDING.mdandScriba.reset/2. Rebuilding a read model is a procedure, not a function: declare the samenamewith a newversion, start it from:originwith its own subscription name, watch:lag_msfall, point reads at the new table, then retire the old version.reset/2is the last step — it clears a version's cursors and watermark (and dead letters on request) so it can start over, refuses while the projection is running, and does not touch the read model, which Scriba does not know the shape of.The guide is explicit about what will bite: handler side effects replay over the whole history, and dead letters are not replayed — a rebuild that ends with the same error kinds fixed nothing. It also records what Scriba will not do: shadow targets and an atomic swap, which are the application's decision, and progress as a percentage, which the source cannot supply because Commanded exposes no head position.
Dead-letter inspection:
Scriba.dead_letters/2lists rows with filters, paging and ordering;Scriba.dead_letter_stats/2summarises how many, of what kind, over what span.Scriba.DeadLetter.list/3,count/3andstats/3are the underlying API.They read the table rather than a running process, and the module form takes the repo from the projection's own config, so a halted or stopped projection can be inspected — which is when anyone actually looks.
The
by_error_kinddistribution is the diagnosis: one kind on one stream is a poison event, one kind across every stream is a schema problem that dead-lettering is papering over.No replay function.
:event_datais stored serialized —__struct__becomes a string — so a row records what failed rather than a value that can be re-dispatched; replay has to read the event from the source by position, and needs an ordering policy this does not yet have.README documents running
broadway_dashboardagainst a projection. It works as-is: a projection is a Broadway topology, and the dashboard both discovers it throughBroadway.all_running/0and accepts Scriba's{:via, Registry, ...}pipeline names. Scriba will not ship a dashboard of its own.:lag_intervalset inuse Scriba.Projectionwas accepted and then ignored — the macro validated it but never passed it on, solag_interval: 0did not disable anything and the Coordinator always used the 5s default. Found by the release audit, between the option shipping and the release.[:scriba, :projection, :lag]telemetry. The Coordinator emits it on a timer —:lag_interval, default 5s,0disables — carryinglag_msand thewatermarkit was derived from, with the projection'sstatusin the metadata.On a timer rather than on traffic, deliberately: a projection that has stopped receiving events emits nothing else, which is exactly when an operator wants to know how far behind it is. It stays silent until the projection has committed something, because a projection with no watermark reporting
lag_ms: 0would read as caught-up when it has not started.Throughput gets no event of its own. Broadway's batch telemetry and the per-event
:stopevents already carry the rate, and a second number could disagree with them.scriba_watermarks: the contiguous global position a projection has reached, surfaced as:watermarkand:lag_msonScriba.info/2. Every event at or below the watermark has been committed, skipped or dead-lettered, with no gap below it — so it is a position a replica could resume from, and the number that says how far a rebuild has got. Per-stream cursors answer neither question: a minimum across them ignores streams the projection never wrote to, a maximum counts work above an event still in flight.:lag_msis measured from the event's ownoccurred_at, not from the event store's head — Commanded's adapter behaviour exposes no head position, so event-count lag is not obtainable, and time-lag is what operators alert on anyway.The source already computed this number to acknowledge safely (0.1.3); this persists it. Written outside the commit transaction and throttled to about one write a second while events are in flight, flushed immediately once the projection catches up — so it can trail what was applied but never runs ahead of it, which is the only direction that is recoverable.
Migration steps are idempotent (
create_if_not_exists). Without that, the documented upgrade path broke fresh installs: an app's original migration callsup(), which means "latest", so a new database got the version 2 table from it and then the upgrade migration collided. Found by runningexamples/bankend to end as part of the release gate.Versioned migrations.
Scriba.Migrations.up/1takes:fromand:to, so a schema change ships as a numbered step: version 1 isscriba_positions+scriba_dead_letters, version 2 addsscriba_watermarks. A fresh install still callsup(); an existing one adds a migration callingup(from: 1). Scriba tracks no migration state of its own — the version lives in the user's migration file, where Ecto already records what ran.Scriba.Testing— run a projection's handlers in a test without a pipeline.project/3applies events through the projection's configured target and commits them, so a test asserts on read-model rows;handle/3calls one clause with no database, for asserting the shape a handler returns. Events run in one transaction, in order, with per-stream cursors advancing as they do in production, and the result reports what committed, skipped, failed, or came back in a shape the target cannot apply.It deliberately stops at the handler and the commit: no retries, no dead-letter routing, no dedup, no telemetry. Those decisions live in the pipeline, and a second implementation of them here would be the copy that drifts.
Changed
- The release gate (architecture §14) now requires the CHANGELOG entry to
account for everything in the release, checked against
git log <previous tag>..HEAD, and records that a published tag is not moved afterwards.
[0.1.4] - 2026-09-16
Documentation only — no code change from 0.1.3.
Changed
Every shipped document audited against the code; 39 claims corrected. README.md, SCRIBA_ARCHITECTURE.md, MIGRATION.md, every
@moduledocand public@docinlib/,examples/bank/README.mdandbench/README.mdwere checked claim by claim against the implementation, with no document treated as evidence for another. Several described the opposite of what the engine does:- A handler
:skipdoes not advance its stream's cursor — the pipeline rejects every:skipfromstream_advances, dedup-induced or not. The README andScriba.Target.Ectosaid it does. - Integrity-class commit failures are dead-lettered, in a transaction
of their own, with
error_kind"commit:<SQLSTATE>".Scriba.TargetandScriba.DeadLettersaid commit failures never are. - A projection also halts when every attempted write in a batch fails
on integrity grounds, and
:halt_reasonis then{:integrity_wipeout, n}rather than aPostgrex.Error. Only structural errors were documented as halting. Scriba.Test.Sourcerequeues failed messages in position order; three places still said it discards them "so the suite cannot replay anything".- Events reach the dead-letter table by five paths, not two: multi-key collisions and unapplicable handler returns bypass the retry layer.
Also corrected: the Coordinator state struct (11 fields, not 7), the halt transition (accepted from every state, not just
:running),Scriba.Info's:halt_reasonfield, the §8/§9 DDL (:utc_datetime_usecistimestamp, nottimestamptz), P1's assertion (the target's commit log, nothandle/2), two dual-emit telemetry sites, the §12 file layout, theScriba.Sourcebehaviour (a GenStage producer plusBroadway.Acknowledger, notBroadway.Producer),:commanded's dependency grouping, the missing:haltedcase inpause/1andresume/1, MIGRATION.md's claim that multi-key collisions raiseArgumentError, andbench/README.md's pre-watermark figures.- A handler
A full documentation audit is now a release gate (architecture §14). Auditing only the documents a change touches is what let the above survive: documents drift against code they never mention.
mix docsnow generates without warnings. Internal modules carry@moduledoc false, so every deliberate mention of one — the telemetry catalog naming its emitters, the architecture guide naming processes, this file naming what changed — produced a "references … but it is hidden" warning.:skip_code_autolink_toinmix.exslists those names, so they render as plain code instead of failing to link. A warning now means a genuinely broken reference.Throughput figures in the README and
Scriba.Source.Commandedre-measured after the 0.1.3 acknowledgement change, on 5,000 events over 100 streams: 9.1 events/sec at the adapter default, 6,002 withbuffer_size: 500.
[0.1.3] - 2026-09-16
Fixed
Acknowledgement advances only across a gapless prefix of committed events. Scriba acknowledged each batch as it committed. Acks are prefix acks — acking event 7 acknowledges everything up to 7 — and batches do not commit in source order when
:parallelismexceeds 1. A handler still working on event 5, or sleeping in the retry loop, did not stop events 6 and 7 from committing and acking, which moved the subscription's checkpoint past event 5. A crash in that window lost event 5 outright: the store believed it had been delivered, the per-stream cursor never advanced, and nothing redelivered it. No dead letter, no cursor anomaly, no log line, andScriba.info/2still reporting:running.The producer now tracks dispatched events in delivery order and acknowledges only the longest run of committed ones with no gap. An uncommitted event holds the watermark back however many later events have committed; a crash then replays from below it, and pipeline-side dedup drops whatever had already been applied.
0.1.2 users should upgrade. That release forwards
:buffer_sizeand its documentation recommends raising it — which is exactly what opens this window. At the adapter default of one in-flight event, no later event can overtake a straggler, so the window is nearly unreachable; withbuffer_size: 500it is wide.Reproduced against a real EventStore in
bench/test/ack_loss_test.exs, which fails on 0.1.2 and passes here.Also faster: one acknowledgement per drain rather than one per event removes a
GenServer.callper event from the commit path. The benchmark went from 2,448 to 4,761 events/sec atbuffer_size: 500.
[0.1.2] - 2026-09-16
Added
:buffer_size,:concurrency_limitand:partition_byare forwarded to the event store subscription.Scriba.Source.Commandedpreviously subscribed with no options at all, so the adapter's defaults always applied.EventStore's default is one in-flight event per subscriber, and that — not:parallelism— is what bounds catch-up: Scriba acknowledges after the batch commits, so a batcher holding a single event waits out its full:batch_timeoutbefore acking and releasing the next.Measured against a real EventStore, 5,000 events over 100 streams: 9.1 events/sec at the adapter default, 2,448 events/sec with
buffer_size: 500.Scriba sets no default of its own — the adapter's still applies unless configured, so this changes nothing for an existing projection until it opts in. Raising the buffer trades memory and redelivered-work-after-a-crash for throughput.
source: {Scriba.Source.Commanded, application: MyApp.CommandedApp, buffer_size: 500}
Fixed
Acknowledgement now comes from the process that holds the subscription. Against
commanded_eventstore_adapter— or any adapter that identifies the acking subscriber byself()— every acknowledgement Scriba issued was silently discarded. The subscription never advanced, and the projection stalled permanently once the event store's in-flight buffer filled. Measured against a real EventStore: one event projected, then nothing, with status still:running, no error, no log line.ack/3is a Broadway acknowledger, so it runs in a batch-processor process rather than the producer that subscribed. It calledack_event/3from there.Commanded.EventStore.Adapters.InMemorytakes the subscription as an argument and ignores the caller, so acks worked;EventStoreresolves the subscriber fromself()and its subscription FSM drops acks from any pid it does not recognise. The engine was correct only against the adapter it was tested with.The batch processor now hands its acknowledgements to the producer, which issues them in delivery order. Both adapters see an ack from a pid they recognise. No public API, configuration, schema or telemetry change.
Anyone running Scriba against a persistent event store should upgrade: on 0.1.0 and 0.1.1 the projection stops after the first batch and does not report that it has.
[0.1.1] - 2026-07-30
Fixed
A halted projection is now queryable, not just observable at the instant it halts.
Scriba.info/2reported:runningfor a projection that had hit a structural commit failure and would never move again — the halt was announced once via[:scriba, :projection, :halted]and a log line, and was invisible from then on. An operator who was not subscribed to telemetry at that moment had a stopped projection and no way to see it, which is the same silent-stall shape the halt path exists to replace.The Pipeline now reports structural failures to the Coordinator, which carries a terminal
:haltedstate.Scriba.info/2exposes it as:statusalong with a new:halt_reasonfield naming the cause (typically aPostgrex.Errorwhose SQLSTATE identifies it). Pollingstatusis enough to detect a stopped projection.pause/1andresume/1are rejected from:halted;stop/1is the way out once the underlying schema or permission is fixed. Additive — no existing field or return value changed.
0.1.0 - 2026-06-10
Initial release. Projection engine for Elixir event-sourced systems
that pairs with Commanded, replacing the role
commanded_ecto_projections
filled before it stopped being actively maintained.
Added
Public API
use Scriba.Projectionmacro. Five-line declaration generates the full projection module:
Compile-time validation rejects unknown options, validatesuse Scriba.Projection, name: "orders", source: {Scriba.Source.Commanded, application: MyApp.CommandedApp}, target: {Scriba.Target.Ecto, repo: MyApp.Repo}, parallelism: 16:partition_byis:stream_id(the only value supported in v0.1), and warns when:namematches the legacy_v<integer>$suffix pattern fromcommanded_ecto_projections.Scriba.start_projection/1and/2. Adds the projection toScriba.Projections.Supervisor(aDynamicSupervisor) at runtime;/2accepts overrides for non-identity options (:parallelism,:retry, etc.) and rejects:name/:versionoverrides since identity is compile-time.Scriba.pause/1,resume/1,stop/1,info/1,list/0. Each accepts either a projection module (reads identity from the generated__scriba_config__/0) or a string name (defaults to version 1).pause/2,resume/2,stop/2,info/2take explicit(name, version)for operator workflows.
Engine internals
- Per-stream cursor tracking in
scriba_positions. One row per(projection_name, projection_version, stream_id)— cross-partition commit ordering can no longer cause silent drops because each stream has its own cursor. - Atomic position + read-model commit. Every successful batch commits
the user's read-model writes AND every touched stream's cursor
advance in one
Ecto.Multitransaction. No observable state where the read model advanced but the cursor didn't, or vice versa. - Consistent-hash partitioning by
stream_idviaScriba.Partitioner.partition/2. Events on the same stream deterministically route to the same Broadway processor; events across streams parallelize up to:parallelism. - Source-side dedup at
Pipeline.handle_message/3. Events whose position is at or below the committed cursor for their stream return:skipwithout invoking the handler — protects against source redelivery after Pipeline restart. - Shared ETS cache for hot-path cursor reads
(
Scriba.Position.Cache). One named table for the whole BEAM, keyed by{name, version, stream_id}tuples. Replaces the earlier per-projection named-table design that allocated one BEAM atom per projection.
Source adapters
Scriba.Source.Commanded. Subscribes to a Commanded.Application viaCommanded.EventStore.subscribe_to/5; convertsCommanded.EventStore.RecordedEventstructs into Broadway messages with%Scriba.Event{}data. Acknowledges back to Commanded after the Multi commits.:commandedis anoptional: truedep — Scriba compiles and runs without it.Scriba.Source.Commanded.pause/1andresume/1callbacks. Pause flips an internal flag;handle_demand/2accumulates demand without dispatching until resume. The EventStore subscription keeps pushing into the source's queue during pause — bounded memory growth documented in the module's moduledoc.
Target adapters
Scriba.Target.Ecto. Builds anEcto.Multifrom per-event handler results plus per-stream cursor advances plus per-event dead-letter inserts; runs everything in a singleRepo.transaction/1. Handler returns become Multi steps::skip→ no step (cursor still advances per-stream).{:insert, struct}→Ecto.Multi.insert/3.{:update, schema, filter, [set: changes]}→Ecto.Multi.update_all/4.{:delete, schema, filter}→Ecto.Multi.delete_all/3.{:multi, %Ecto.Multi{}}→ merged viaEcto.Multi.merge/2.
Scriba.Target.Test. In-memoryAgent-backed target for property tests and downstream user testing. Records commits, per-stream positions, and dead-letter entries.
Operational features
- Telemetry events (full surface in
Scriba.Telemetry's moduledoc):[:scriba, :projection, :event, :start | :stop | :exception]— via:telemetry.span/3around each handler invocation. Fires per retry attempt.[:scriba, :projection, :batch, :stop]— manual emit after Multi commit succeeds. Measurements%{duration, batch_size}.[:scriba, :projection, :dead_letter]— once per dead-lettered event after the Multi commits. Metadata%{projection, position, stream_id, event_type, error_kind}.[:scriba, :projection, :started | :paused | :resumed]— Coordinator lifecycle events.:startedfires once per Coordinator-process lifetime (Pipeline DOWN → re-up does NOT re-fire).[:scriba, :projection, :cache_initialized]— ETS preload from Postgres at Coordinator start.[:scriba, :source, :batch, :failed]— a batch failed to commit and nothing in it was acknowledged. Measurements%{count}, metadata%{subscription, reason}.[:scriba, :projection, :halted]— a structural commit failure stopped the projection. Metadata%{projection, reason, failure}wherefailureis the SQLSTATE label. The event to page on.
- Commit failures are classified by SQLSTATE, not guessed
(
Scriba.Failure).:integrity(classes 22/23,Ecto.ConstraintError, invalid changesets) is deterministic and event-specific — the batch is re-applied one transaction per event so the offender dead-letters witherror_kind"commit:<SQLSTATE>"and the rest commit.:transient(classes 08/53,40001,40P01,57014,DBConnectionerrors) replays.:structural(class 42 and anything unrecognised) halts the projection and emits[:scriba, :projection, :halted], because replaying loops forever and dead-lettering would destroy a batch over a fixable deploy-ordering mistake. In the per-event pass a stream stops at its first unresolved event rather than skipping past it, preserving per-stream ordering and preventing a cursor gap. - Batch commit failures replay; they never partially acknowledge.
When a target's transaction does not commit, nothing in the batch is
acknowledged — including the messages that succeeded, because event
store acks are prefix-acks and cannot express a gap. The source stops
its producer, which rewinds the subscription to its last durable
checkpoint; the batch is redelivered and source-side dedup filters
whatever did commit. See
Scriba.BatchCommitError. - Malformed handler returns dead-letter instead of wedging the
projection. A return outside §4.2's six shapes is routed to
scriba_dead_letterswitherror_kind"invalid_return"and the cursor advances. Targets declare their own vocabulary through the optionalScriba.Target.valid_result?/1callback, so this does not hard-code the Ecto target's shapes into the engine. - Duplicate
Ecto.Multioperation names within a batch are detected before assembly. Scriba merges a whole batch into one Multi, so two events naming an operation identically would collide insideRepo.transaction/1— deterministically, on every redelivery. The first claim now wins and later claimants dead-letter individually witherror_kind"multi_key_collision". Scriba's own step keys ({:scriba_event, _},{:scriba_position, _},{:scriba_dead_letter, _}) are reserved. - Cursor monotonicity is enforced in the schema. The
scriba_positionsupsert isGREATEST(existing, incoming), so a cursor cannot move backwards regardless of what the Pipeline computes. - Restart intensity is chosen, not defaulted.
Scriba.Projection.Supervisorruns at 30 restarts per 60 seconds. A commit failure now kills the producer on purpose, so the OTP default of 3-in-5 would exhaust during any real outage and propagate toward the host application; 30-in-60 absorbs a multi-minute outage while still bounding a genuine crash loop. - Resubscribe tolerates the reap race. After a producer dies to force
a replay, the event store may not yet have processed its
DOWN, so resubscribing can transiently see:subscription_already_exists. That is now retried with backoff (~1.5s total) and, if it persists, reported as a name conflict rather than as a migration problem. - Dead-letter routing. Handler
{:error, _}returns and raised exceptions route toscriba_dead_lettersafter retry exhaustion, with the cursor advancing past the failed event (skip-and-continue per architecture §9.2). The dead-letter insert and the cursor advance commit atomically in the same Multi as the batch's successful writes. - Retry policy with exponential backoff (default 3 attempts at
100ms / 1s / 10s) configurable per projection via
retry: [max_attempts: N, backoff: [...]]inuse Scriba.Projection.retry: falseopts out for immediate dead-letter. Implementation is an in-handlerProcess.sleeploop — seeSCRIBA_ARCHITECTURE.md§9 for why this is the right primitive rather thanProcess.send_after/3against Broadway's processor model. - Real pause / resume with held demand. The Coordinator's
:running → :pausedtransition signals the source's GenStage producer to stop yielding; the Pipeline tree stays alive, in-flight events finish their commit lifecycle, accumulated demand drains on resume. Replaces the earlierterminate_child/restart_childimplementation that was misleadingly named. - Scriba attaches no telemetry handlers of its own. It emits events
and gets out of the way; users attach their own in their app's
start/2. Default handlers, if any, are a v0.2 concern and will arrive with the behaviour that justifies them.
Migrations
Scriba.Migrations.up/0anddown/0. Users call these from their own Ecto migration:
Createsdefmodule MyApp.Repo.Migrations.AddScribaTables do use Ecto.Migration def up, do: Scriba.Migrations.up() def down, do: Scriba.Migrations.down() endscriba_positions(per-stream cursors) andscriba_dead_letters.
Documentation
MIGRATION.md— migrating fromcommanded_ecto_projections.project/2,3→handle/2, theEcto.Multidifferences (one transaction per batch, not per event), themetamap mapping, callbacks with no equivalent (after_update/3,schema_prefix/1,consistency: :strong), and cursor carry-over: both libraries track Commanded's globalevent_number, solast_seen_event_numberpassed as:start_fromcuts over in place with no read-model rebuild.:start_fromexclusivity verified againstcommanded_eventstore_adapter+eventstore.- Legacy
use Scriba.Projectionoptions raise with targeted guidance rather than a generic unknown-option error —:consistencymost of all, since it is the only migration gap that is invisible at runtime. SCRIBA_ARCHITECTURE.md— the architectural contract. Non-negotiable invariants, supervision tree, Coordinator state machine, position-tracking storage shape, error handling, property tests, dependencies, file layout, behaviour signatures.- Module-level moduledocs on every public module documenting the
contract from the user's perspective, including sharp edges
(
Scriba.Source.Commanded's pause-memory caveat,Scriba.Target.Ecto's{:multi, _}escape hatch for SQL-level increments, etc.).
Example application
examples/bank/— self-contained Mix project demonstrating the five-line API end-to-end. Real Commanded (Commanded.EventStore.Adapters.InMemoryfor fast iteration), real Ecto, real read model. Three accounts, fifty random deposits/withdrawals, the projection converges and the demo prints the final balances.mix bank.setup+mix bank.demoruns in under 30 seconds against a local Postgres.
Fixed
Scriba.Source.Commanded.to_message/2read the wrong event-struct field. It readstream_uuidfrom Commanded'sRecordedEvent, but the field in Commanded 1.4 isstream_id. Any event delivered from a real Commanded subscription would have raisedKeyErroron the first message. The bug was present in the initialSource.Commandedimplementation and went undetected because that path had no end-to-end coverage until the Commanded integration tests were added during pre-release verification (commit2778ae9). It never shipped in a tagged release — fixed before v0.1.0. Anyone evaluating Scriba against Commanded from a pre-release snapshot who hits astream_uuidKeyErrorshould move to v0.1.0.
Property tests
The three architectural invariants from SCRIBA_ARCHITECTURE.md §10
are mandatory for v0.1 and covered by property tests:
- P1 — Per-stream ordering. The sequence of events delivered to
handle/2for any stream is a prefix of the source order for that stream. - P2 — Position monotonicity. Committed positions in Postgres are monotonically non-decreasing across all observed states, including across simulated worker crashes (200 iterations against real Postgres).
- P3 — Effectively-once under crash. For each event, the handler's
effect appears in the target exactly once after recovery from
arbitrary crash schedules (100 iterations against real Postgres for
the cursor-resume variant; the integration-test side of the
property is in
test/scriba/projection/pipeline_test.exs).
Dependencies
:broadway~> 1.1:ecto_sql~> 3.11:postgrex~> 0.17:telemetry~> 1.2:jason~> 1.4:commanded~> 1.4(optional)
Dev / test:
:stream_data,:ex_doc,:credo,:dialyxir.
Architectural decisions worth knowing
These are documented at length in SCRIBA_ARCHITECTURE.md but worth
calling out for users upgrading from commanded_ecto_projections or
similar:
nameandversionare separate fields. Do not encode the version in the name ("orders_v1"is wrong; the macro warns at compile time). The architecture supports running multiple versions side-by-side during a cutover via two modules with differentversion:integers.- Dead-lettered events advance the cursor. Blocking the projection on a bad event is the #2 complaint on ElixirForum after lag visibility (per architecture §9.2). Scriba defaults to skip-and-continue with dead-letter visibility via telemetry; users who need block-on-failure semantics build that on top.
- No idempotency on
pause/resume.pauseon:pausedandresumeon:runningreturn{:error, {:invalid_state, _}}, not:ok. Silent idempotency hides bugs. Callers wanting "make sure this is paused" semantics checkScriba.info/1first or pattern-match the matching-state error case as success. - Multi transaction failures do not retry through the per-event retry policy. Retry wraps the handler call, not the Multi commit. Database-level failure leaves the batch unacked; the source re-delivers when the projection's Pipeline next runs.