Extreme.PersistentSubscription (extreme v1.1.2)
View SourceAn asynchronous subscription strategy.
Other subscription methods require stream positions to be persisted by the
client (e.g. in :dets
or PostgreSQL). Persistent Subscription is a
subscription strategy in which details about backpressure, buffer sizes, and
stream positions are all held by the EventStore (server).
In a persistent subscription, all communication is asynchronous. When an
event is received and processed, it must be acknowledged as processed by
ack/3
in order to be considered by the server as processed. The server
stores knowledge of which events have been processed by means of checkpoints,
so listeners which use persistent subscriptions do not store stream positions
themselves.
The system of ack/3
s and nack/5
s allows listeners to handle events in
unconventional ways:
- concurrent processing: events may be handled by multiple processors at the same time.
- out of order processing: events may be handled in any order.
- retry: events which are
nack/5
-ed with the:retry
action, and events which do not receive acknowledgement viaack/3
are retried. - message parking: if an event is not acknowledged and reaches its maximum retry count, the message is parked in a parked messages queue. This prevents head-of-line blocking typical of other subscription patterns.
- competing consumers: multiple consumers may process events without collaboration or gossip between the consumers themselves.
Persistent subscriptions are started with
Extreme.connect_to_persistent_subscription/4
expect a cast
of each event
in the form of {:on_event, event, correlation_id}
A Persistent Subscription must exist before it can be connected to.
Persistent Subscriptions can be created by sending the
Extreme.Messages.CreatePersistentSubscription
message via
Extreme.execute/3
, by the HTTP API, or in the EventStore dashboard.
Example
defmodule MyPersistentListener do
use GenServer
alias Extreme.PersistentSubscription
def start_link(opts) do
GenServer.start_link(__MODULE__, opts)
end
def init(opts) do
{:ok, opts}
end
def subscribe(listener_proc), do: GenServer.cast(listener_proc, :subscribe)
def handle_cast(:subscribe, state) do
{:ok, subscription_pid} =
MyExtremeClientModule.connect_to_persistent_subscription(
self(),
opts.stream,
opts.group,
opts.allowed_in_flight_messages
)
{:noreply, Map.put(state, :subscription_pid, subscription_pid)}
end
def handle_cast({:on_event, event, correlation_id}, state) do
# .. do the real processing here ..
:ok = PersistentSubscription.ack(state.subscription_pid, event, correlation_id)
{:noreply, state}
end
def handle_call(:unsubscribe, _from, state) do
:ok = MyExtremeClientModule.unsubscribe(state.subscription_pid)
{:reply, :ok, state}
end
def handle_info(_, state), do: {:noreply, state}
end
Summary
Functions
Acknowledges that an event or set of events have been successfully processed.
Returns a specification to start this module under a supervisor.
Acknowledges that an event or set of events could not be handled.
Types
An event received from a persistent subscription.
@type event_id() :: binary()
An event ID.
Either from the :link
or :event
, depending on if the event is from a
projection stream or a normal stream (respectively).
Functions
Acknowledges that an event or set of events have been successfully processed.
ack/3
takes any of the following for event ID:
- a full event, as given in the
{:on_event, event, correlation_id}
cast - the
event_id
of an event (either from its:link
or:event
, depending on if the event comes from a projection or a normal stream, respectively) - a list of either sort
correlation_id
comes from the :on_event
cast.
Example
def handle_cast({:on_event, event, correlation_id}, state) do
# .. do some processing ..
# when the processing completes successfully:
:ok = Extreme.PersistentSubscription.ack(state.subscription_pid, event, correlation_id)
{:noreply, state}
end
Returns a specification to start this module under a supervisor.
See Supervisor
.
@spec nack( pid(), event() | event_id() | [event() | event_id()], binary(), :unknown | :park | :retry | :skip | :stop, String.t() ) :: :ok
Acknowledges that an event or set of events could not be handled.
See ack/3
for information on event
and correlation_id
.
action
can be any of the following
:unknown
:park
:retry
:skip
:stop
The :park
action sets aside the event in the Parked Messages queue, which
may be replayed via the HTTP
API
or by button click in the EventStore Persistent Subscriptions dashboard.
When an event reaches the max retry count configured by the
:max_retry_count
field in Extreme.Messages.CreatePersistentSubscription
,
the event is parked.
Example
def handle_cast({:on_event, event, correlation_id}, state) do
# .. do some processing ..
# in the case that the processing fails and should be retried:
:ok = Extreme.PersistentSubscription.nack(state.subscription_pid, event, correlation_id, :retry)
{:noreply, state}
end