Exmld.KinesisWorker.Datum (exmld v1.0.2) View Source

Link to this section Summary

Functions

Struct for annotating stream records (or heartbeats) with additional data.

Link to this section Types

Specs

t() :: %Exmld.KinesisWorker.Datum{
  opaque: any(),
  shard_id: Exmld.shard_id(),
  stream_record: Exmld.stream_record() | {:heartbeat, any()}
}

Link to this section Functions

Link to this function

%Exmld.KinesisWorker.Datum{}

View Source (struct)

Struct for annotating stream records (or heartbeats) with additional data.

To allow a single downstream processing pipeline to be used with multiple source streams, we annotate Kinesis stream records with additional data before providing them to the stage(s) processing them. (Example: :opaque could be a 2-tuple naming the source stream and region or otherwise indicate how to specially process the record).

Fields

  • :opaque - the opaque term provided at worker init time
  • :shard_id - name of the shard the worker is processing
  • :stream_record - a record from the stream or {:heartbeat, _}