Exmld.KinesisWorker.Datum (exmld v1.0.2) View Source
Link to this section Summary
Functions
Struct for annotating stream records (or heartbeats) with additional data.
Link to this section Types
Specs
t() :: %Exmld.KinesisWorker.Datum{ opaque: any(), shard_id: Exmld.shard_id(), stream_record: Exmld.stream_record() | {:heartbeat, any()} }
Link to this section Functions
Struct for annotating stream records (or heartbeats) with additional data.
To allow a single downstream processing pipeline to be used with multiple source
streams, we annotate Kinesis stream records with additional data before providing them
to the stage(s) processing them. (Example: :opaque
could be a 2-tuple naming the
source stream and region or otherwise indicate how to specially process the record).
Fields
:opaque
- the opaque term provided at worker init time:shard_id
- name of the shard the worker is processing:stream_record
- a record from the stream or{:heartbeat, _}