kafka_gen_stage v2.0.2 KafkaGenStage.ConsumerLogic View Source
Separates pure logic of consumer genstage to be unit tested.
Link to this section Summary
Types
Acking to prod consumer
Last offset (inclusive) to be emmited by GenStage
Format of read messages
Functions
Enqueue incoming messages up to end_offset(inclusive)
Handles caching of both, demand and events, as recomended in Buffering Demand in gen_stage docs.
Thus called from both, KafkaGenStage.Consumer.handle_demand/2
and
KafkaGenStage.Consumer.handle_info/2
where events from brod_consumer arrive
Link to this section Types
Acking to prod consumer.
Last offset (inclusive) to be emmited by GenStage.
msg_tuple() :: {offset :: non_neg_integer(), timestamp :: non_neg_integer(), key :: binary(), value :: binary()}
Format of read messages.
Link to this section Functions
enqueue( queue :: :queue.queue(msg_tuple()), messages :: Enum.t(), inclusive_end_offset :: end_offset() ) :: :queue.queue(msg_tuple())
Enqueue incoming messages up to end_offset(inclusive)
prepare_dispatch( buffered_msgs :: :queue.queue(), buffered_demand :: non_neg_integer(), bulk_transformer :: ([msg_tuple()] -> [msg_tuple()]), is_end_of_stream :: boolean() ) :: {msgs_to_send :: [msg_tuple()], ack_to_brod_consumer :: ack(), buffered_demand :: non_neg_integer(), buffered_msgs :: :queue.queue()}
Handles caching of both, demand and events, as recomended in Buffering Demand in gen_stage docs.
Thus called from both, KafkaGenStage.Consumer.handle_demand/2
and
KafkaGenStage.Consumer.handle_info/2
where events from brod_consumer arrive.