Spear.stream-exclamation-mark
stream-exclamation-mark
, go back to Spear module for more information.
Specs
stream!( connection :: Spear.Connection.t(), stream_name :: String.t() | :all, opts :: Keyword.t() ) :: event_stream :: Enumerable.t()
Collects an EventStoreDB stream into an enumerable
This function may raise in cases where the gRPC requests fail to read events from the EventStoreDB (in cases of timeout or unavailability).
This function does not raise if a stream does not exist (is empty), instead
returning an empty enumerable []
.
connection
may be any valid GenServer name (including PIDs) for a process
running the Spear.Connection
GenServer.
stream_name
can be any stream, existing or not, including projected
streams such as category streams or event-type streams. The :all
atom
may be passed as stream_name
to read all events in the EventStoreDB.
Options
:from
- (default::start
) the EventStoreDB stream revision from which to read. Valid values include:start
,:end
, any non-negative integer representing the event number revision in the stream and events. Event numbers are inclusive (e.g. reading from0
will first return the event with revision0
in the stream, if one exists).:start
and:end
are treated as inclusive (e.g.:start
will return the first event in the stream). Events (eitherSpear.Event
or ReadResp records) can also be supplied and will be treated as inclusive.:direction
- (default::forwards
) the direction in which to read the EventStoreDB stream. Valid values include:forwards
and:backwards
. Reading the EventStoreDB stream forwards will return events in the order in which they were written to the EventStoreDB; reading backwards will return events in the opposite order.:resolve_links?
- (default:true
) whether or not to request that link references be resolved. See the moduledocs for more information about link resolution.:chunk_size
- (default:128
) the number of events to read from the EventStoreDB at a time. Any positive integer is valid. See the enumeration characteristics section below for more information about how:chunk_size
works and how to tune it.:timeout
- (default:5_000
- 5s) the time allowed for the read of a single chunk of events in the EventStoreDB stream. This time is not cumulative: an EventStoreDB stream 100 events long which takes 5s to read each chunk may be read in chunks of 20 events culumaltively in 25s. A timeout of5_001
ms would not raise a timeout error in that scenario (assuming the chunk read consistently takes<= 5_000
ms).:raw?:
- (default:false
) controls whether or not the enumerableevent_stream
is decoded toSpear.Event
structs from their rawSpear.Records.Streams.read_resp/0
output. Settingraw?: true
prevents this transformation and leaves each event as aReadResp
record. SeeSpear.Event.from_read_response/2
for more information.:credentials
- (default:nil
) a two-tuple{username, password}
to use as credentials for the request. This option overrides any credentials set in the connection configuration, if present. See the Security guide for more details.
Enumeration Characteristics
The event_stream
Enumerable.t/0
returned by this function initially
contains a buffer of bytes from the first read of the stream stream_name
.
This buffer potentially contains up to :chunk_size
messages when run.
The enumerable is written as a formula which abstracts away the chunking
nature of the gRPC requests, however, so even though the EventStoreDB stream is
read in chunks (per the :chunk_size
option), the entire EventStoreDB stream
can be read by running the enumeration (e.g. with Enum.to_list/1
). Note
that the stream will make a gRPC request to read more events whenever the
buffer runs dry with up to :chunk_size
messages filling the buffer on each
request.
:chunk_size
is difficult to tune as it causes a tradeoff between (gRPC)
request duration and number of messages added to the buffer. A higher
:chunk_size
may hydrate the buffer with more events and reduce the number
of gRPC requests needed to read an entire stream, but it also increases
the number of messages that will be sent over the network per request which
could decrease reliability. Generally speaking, a lower :chunk_size
is
appropriate for streams in which the events are large and a higher
:chunk_size
is appropriate for streams with many small events. Manual
tuning and trial-and-error can be used to find a performant :chunk_size
setting for any individual environment.
Examples
iex> Spear.stream!(MyConnection, "es_supported_clients", chunk_size: 1) |> Enum.take(1)
[
%Spear.Event{
body: %{"languages" => ["typescript", "javascript"], "runtime" => "NodeJS"},
id: "1fc908c1-af32-4d06-a9bd-3bf86a833fdf",
metadata: %{..},
type: "grpc-client"
}
]
# say we have 5 events in the "es_supported_clients" stream
iex> Spear.stream!(MyConnection, "es_supported_clients", chunk_size: 3) |> Enum.count()
5