extreme v0.13.3 Extreme View Source
Extreme module is main communication point with EventStore using tcp connection. Extreme is implemented using GenServer and is OTP compatible. If client is disconnected from server we are not trying to reconnect, instead you should rely on your supervisor. For example:
defmodule MyApp.Supervisor do
use Supervisor
def start_link,
do: Supervisor.start_link __MODULE__, :ok
@event_store MyApp.EventStore
def init(:ok) do
event_store_settings = Application.get_env :my_app, :event_store
children = [
worker(Extreme, [event_store_settings, [name: @event_store]]),
# ... other workers / supervisors
]
supervise children, strategy: :one_for_one
end
end
You can manually start adapter as well (as you can see in test file):
{:ok, server} = Application.get_env(:extreme, :event_store) |> Extreme.start_link
From now on, server
pid is used for further communication. Since we are relying on supervisor to reconnect,
it is wise to name server
as we did in example above.
Link to this section Summary
Functions
Cast the provided value to an atom if appropriate. If the provided value is a string, convert it to an atom, otherwise return it as-is
Returns a specification to start this module under a supervisor
Connect the subscriber
to an existing persistent subscription named subscription
on stream
Executes protobuf message
against server
. Returns
Invoked when the server is started. start_link/3
or start/3
will
block until it returns
Reads events specified in read_events
, sends them to subscriber
and leaves subscriber
subscribed per subscribe
message
Starts connection to EventStore using connection_settings
and optional opts
Subscribe subscriber
to stream
using server
Link to this section Functions
Cast the provided value to an atom if appropriate. If the provided value is a string, convert it to an atom, otherwise return it as-is.
Returns a specification to start this module under a supervisor.
See Supervisor
.
Connect the subscriber
to an existing persistent subscription named subscription
on stream
subscriber
is process that will keep receiving {:on_event, event} messages.
Returns {:ok, subscription} when subscription is success.
Executes protobuf message
against server
. Returns:
- {:ok, protobuf_message} on success .
- {:error, :not_authenticated} on wrong credentials.
- {:error, error_reason, protobuf_message} on failure.
EventStore uses ProtoBuf for taking requests and sending responses back.
We are using exprotobuf to deal with them.
List and specification of supported protobuf messages can be found in include/event_store.proto
file.
Instead of wrapping each and every request in elixir function, we are using execute/2
function that takes server pid and request message:
{:ok, response} = Extreme.execute server, write_events()
where write_events
can be helper function like:
alias Extreme.Msg, as: ExMsg
defp write_events(stream \ "people", events \ [%PersonCreated{name: "Pera Peric"}, %PersonChangedName{name: "Zika"}]) do
proto_events = Enum.map(events, fn event ->
ExMsg.NewEvent.new(
event_id: Extreme.Tools.gen_uuid(),
event_type: to_string(event.__struct__),
data_content_type: 0,
metadata_content_type: 0,
data: :erlang.term_to_binary(event),
meta: ""
) end)
ExMsg.WriteEvents.new(
event_stream_id: stream,
expected_version: -2,
events: proto_events,
require_master: false
)
end
This way you can fine tune your requests, i.e. choose your serialization. We are using erlang serialization in this case
data: :erlang.term_to_binary(event)
, but you can do whatever suites you.
For more information about protobuf messages EventStore uses,
take a look at their documentation or for common use cases
you can check test/extreme_test.exs
file.
Invoked when the server is started. start_link/3
or start/3
will
block until it returns.
args
is the argument term (second argument) passed to start_link/3
.
Returning {:ok, state}
will cause start_link/3
to return
{:ok, pid}
and the process to enter its loop.
Returning {:ok, state, timeout}
is similar to {:ok, state}
except handle_info(:timeout, state)
will be called after timeout
milliseconds if no messages are received within the timeout.
Returning {:ok, state, :hibernate}
is similar to {:ok, state}
except the process is hibernated before entering the loop. See
c:handle_call/3
for more information on hibernation.
Returning {:ok, state, {:continue, continue}}
is similar to
{:ok, state}
except that immediately after entering the loop
the c:handle_continue/2
callback will be invoked with the value
continue
as first argument.
Returning :ignore
will cause start_link/3
to return :ignore
and
the process will exit normally without entering the loop or calling
c:terminate/2
. If used when part of a supervision tree the parent
supervisor will not fail to start nor immediately try to restart the
GenServer
. The remainder of the supervision tree will be started
and so the GenServer
should not be required by other processes.
It can be started later with Supervisor.restart_child/2
as the child
specification is saved in the parent supervisor. The main use cases for
this are:
- The
GenServer
is disabled by configuration but might be enabled later. - An error occurred and it will be handled by a different mechanism than the
Supervisor
. Likely this approach involves callingSupervisor.restart_child/2
after a delay to attempt a restart.
Returning {:stop, reason}
will cause start_link/3
to return
{:error, reason}
and the process to exit with reason reason
without
entering the loop or calling c:terminate/2
.
Callback implementation for GenServer.init/1
.
Reads events specified in read_events
, sends them to subscriber
and leaves subscriber
subscribed per subscribe
message.
subscriber
is process that will keep receiving {:on_event, event} messages.
read_events
:: Extreme.Msg.ReadStreamEvents
subscribe
:: Extreme.Msg.SubscribeToStream
Returns {:ok, subscription} when subscription is success.
If stream
is hard deleted subscriber
will receive message {:extreme, :error, :stream_hard_deleted, stream}
If stream
is soft deleted subscriber
will receive message {:extreme, :warn, :stream_soft_deleted, stream}.
In case of soft deleted stream, new event will recreate stream and it will be sent to subscriber
as described above
Hard deleted streams can’t be recreated so suggestion is not to handle this message but rather crash when it happens
Examples:
defmodule MyApp.StreamSubscriber
use GenServer
def start_link(extreme, last_processed_event),
do: GenServer.start_link __MODULE__, {extreme, last_processed_event}
def init({extreme, last_processed_event}) do
stream = "people"
state = %{ event_store: extreme, stream: stream, last_event: last_processed_event }
GenServer.cast self(), :subscribe
{:ok, state}
end
def handle_cast(:subscribe, state) do
# read only unprocessed events and stay subscribed
{:ok, subscription} = Extreme.read_and_stay_subscribed state.event_store, self(), state.stream, state.last_event + 1
# we want to monitor when subscription is crashed so we can resubscribe
ref = Process.monitor subscription
{:noreply, %{state|subscription_ref: ref}}
end
def handle_info({:DOWN, ref, :process, _pid, _reason}, %{subscription_ref: ref} = state) do
GenServer.cast self(), :subscribe
{:noreply, state}
end
def handle_info({:on_event, push}, state) do
push.event.data
|> :erlang.binary_to_term
|> process_event
event_number = push.link.event_number
:ok = update_last_event state.stream, event_number
{:noreply, %{state|last_event: event_number}}
end
def handle_info(_msg, state), do: {:noreply, state}
defp process_event(event), do: IO.puts("Do something with event: " <> inspect(event))
defp update_last_event(_stream, _event_number), do: IO.puts("Persist last processed event_number for stream")
end
This way unprocessed events will be sent by Extreme, using {:on_event, push}
message.
After all persisted messages are sent, new messages will be sent the same way as they arrive to stream.
Since there’s a lot of boilerplate code here, you can use Extreme.Listener
to reduce it and focus only
on business part of code.
Starts connection to EventStore using connection_settings
and optional opts
.
Extreme can connect to single ES node or to cluster specified with node IPs and ports.
Example for connecting to single node:
config :extreme, :event_store,
db_type: :node,
host: "localhost",
port: 1113,
username: "admin",
password: "changeit",
reconnect_delay: 2_000,
connection_name: :my_app,
max_attempts: :infinity
db_type
- defaults to :node, thus it can be omittedhost
- check EXT IP setting of your EventStoreport
- check EXT TCP PORT setting of your EventStorereconnect_delay
- in ms. Defaults to 1_000. If tcp connection fails this is how long it will wait for reconnection.connection_name
- Optional param introduced in EventStore 4. Connection can be identified by this name on ES UImax_attempts
- Defaults to :infinity. Specifies how many times we’ll try to connect to EventStore
Example for connecting to cluster:
config :extreme, :event_store,
db_type: :cluster,
gossip_timeout: 300,
nodes: [
%{host: "10.10.10.29", port: 2113},
%{host: "10.10.10.28", port: 2113},
%{host: "10.10.10.30", port: 2113}
],
connection_name: :my_app,
username: "admin",
password: "changeit"
gossip_timeout
- in ms. Defaults to 1_000. We are iterating throughnodes
list, asking for cluster member details. This setting represents timeout for gossip response before we are asking next node fromnodes
list for cluster details.nodes
- Mandatory for cluster connection. Represents list of nodes in the cluster as we know ithost
- should be EXT IP setting of your EventStore nodeport
- should be EXT HTTP PORT setting of your EventStore node
Example of connection to cluster via DNS lookup
config :extreme, :event_store,
db_type: :cluster_dns,
gossip_timeout: 300,
host: "es-cluster.example.com", # accepts char list too, this whould be multy A record host enrty in your nameserver
port: 2113, # the external gossip port
connection_name: :my_app,
username: "admin",
password: "changeit",
max_attempts: :infinity
When cluster
mode is used, adapter goes thru nodes
list and tries to gossip with node one after another
until it gets response about nodes. Based on nodes information from that response it ranks their statuses and chooses
the best candidate to connect to. For the way ranking is done, take a look at lib/cluster_connection.ex
:
defp rank_state("Master"), do: 1
defp rank_state("PreMaster"), do: 2
defp rank_state("Slave"), do: 3
defp rank_state("Clone"), do: 4
defp rank_state("CatchingUp"), do: 5
defp rank_state("PreReplica"), do: 6
defp rank_state("Unknown"), do: 7
defp rank_state("Initializing"), do: 8
Note that above will work with same procedure with cluster_dns
mode turned on, since internally it will get ip addresses to witch same connection procedure will be used.
Once client is disconnected from EventStore, supervisor should respawn it and connection starts over again.
Subscribe subscriber
to stream
using server
.
subscriber
is process that will keep receiving {:on_event, event} messages.
Returns {:ok, subscription} when subscription is success.
NOTE: If `stream` is hard deleted, `subscriber` will NOT receive any message!
Example:
def subscribe(server, stream \ "people"), do: Extreme.subscribe_to(server, self(), stream)
def handle_info({:on_event, event}, state) do
Logger.debug "New event added to stream 'people': " <> inspect(event)
{:noreply, state}
end
As Extreme.read_and_stay_subscribed/7
has it’s abstraction in Extreme.Listener
, there’s abstraction for this function
as well in Extreme.FanoutListener
behaviour.