kafka_gen_stage v2.0.1 KafkaGenStage.ConsumerLogic View Source
Separates pure logic of consumer genstage to be unit tested.
Link to this section Summary
Types
Acking to prod consumer
Internal type for working with demand and event buffers
Last offset (inclusive) to be emmited by GenStage
Format of read messages
API type for working with demand and event buffers
Functions
Insert incoming messages into buffer, up to end_offset(inclusive)
Handles caching of both, demand and events, as recomended in Buffering Demand in gen_stage docs.
Thus called from both, KafkaGenStage.Consumer.handle_demand/2
and
KafkaGenStage.Consumer.handle_info/2
where events from brod_consumer arrive
Link to this section Types
Acking to prod consumer.
dispatch() :: {msgs_to_send :: [msg_tuple()], ack_to_brod_consumer :: ack(), buffered_msgs :: :queue.queue()}
Internal type for working with demand and event buffers.
Last offset (inclusive) to be emmited by GenStage.
msg_tuple() :: {offset :: non_neg_integer(), timestamp :: non_neg_integer(), key :: binary(), value :: binary()}
Format of read messages.
return_dispatch() :: {msgs_to_send :: [msg_tuple()], ack_to_brod_consumer :: ack(), buffered_demand :: non_neg_integer(), buffered_msgs :: :queue.queue()}
API type for working with demand and event buffers.
Link to this section Functions
messages_into_queue( messages :: [any()], buffered_message :: :queue.queue(msg_tuple()), inclusive_end_offset :: end_offset(), (msg :: term() -> msg_tuple()) ) :: {:cont | :halt, :queue.queue(msg_tuple())}
Insert incoming messages into buffer, up to end_offset(inclusive)
prepare_dispatch( buffered_msgs :: :queue.queue(), buffered_demand :: non_neg_integer(), bulk_transformer :: ([msg_tuple()] -> [msg_tuple()]), is_end_of_stream :: boolean() ) :: return_dispatch()
Handles caching of both, demand and events, as recomended in Buffering Demand in gen_stage docs.
Thus called from both, KafkaGenStage.Consumer.handle_demand/2
and
KafkaGenStage.Consumer.handle_info/2
where events from brod_consumer arrive.