exmld v0.1.6 Exmld.KinesisWorker.Datum View Source

Link to this section Summary

Functions

Struct for annotating stream records (or heartbeats) with additional data

Link to this section Types

Link to this type t() View Source
t() :: %Exmld.KinesisWorker.Datum{opaque: any(), shard_id: Exmld.shard_id(), stream_record: Exmld.stream_record() | {:heartbeat, any()}}

Link to this section Functions

Link to this function %Exmld.KinesisWorker.Datum{} View Source (struct)

Struct for annotating stream records (or heartbeats) with additional data.

To allow a single downstream processing pipeline to be used with multiple source streams, we annotate Kinesis stream records with additional data before providing them to the stage(s) processing them. (Example: :opaque could be a 2-tuple naming the source stream and region or otherwise indicate how to specially process the record).

Fields

  • :opaque - the opaque term provided at worker init time
  • :shard_id - name of the shard the worker is processing
  • :stream_record - a record from the stream or {:heartbeat, _}