KafkaBatcher.Accumulator.State (kafka_batcher v1.0.1)

Encapsulates all logic to detect when batch will be ready to producing

A batch is marked as ready for producing when one of the following conditions is met:

  • reached the max byte size of the batch
  • reached the batch size (messages count) limit
  • reached the waiting time limit (max delay before producing)
  • one of special events arrived (which triggers Flusher to produce immediately)
  • timer expired (in case when a few events arrived timer helps to control that the max waiting time is not exceeded)

Summary

Types

@type t() :: %KafkaBatcher.Accumulator.State{
  batch_bytesize: non_neg_integer(),
  batch_flusher: atom(),
  batch_size: non_neg_integer(),
  cleanup_timer_ref: reference() | nil,
  collector: atom() | nil,
  config: Keyword.t(),
  last_produced_at: non_neg_integer(),
  max_batch_bytesize: non_neg_integer(),
  max_wait_time: non_neg_integer(),
  messages_to_produce: list(),
  min_delay: non_neg_integer(),
  partition: non_neg_integer() | nil,
  pending_messages: list(),
  pending_messages_count: non_neg_integer(),
  producer_config: Keyword.t(),
  status: atom(),
  topic_name: binary()
}

Functions

Link to this function

add_new_message(state, event, now)

@spec add_new_message(t(), KafkaBatcher.MessageObject.t(), non_neg_integer()) :: t()
Link to this function

mark_as_ready(state)

@spec mark_as_ready(t()) :: t()
Link to this function

reset_state_after_failure(state)

@spec reset_state_after_failure(t()) :: t()
Link to this function

reset_state_after_produce(state)

@spec reset_state_after_produce(t()) :: t()