View Source OffBroadway.Kafka.Producer (off_broadway_kafka_pipeline v2.0.0)

Defines a Broadway Producer module which receives messages from Kafka to initiate the Broadway pipeline.

Sends messages to the handle_info/2 and handle_demand/2 functions based on requests and tracks acknowledgements in state.

Link to this section Summary

Functions

Passes Kafka partition assignment changes to the handle_info/2 function.

Handles message events based on demand.

Handles assignments revoked by the Kafka broker.

Passes Kafka messages to the handle_info/2 function.

Names the producer process and initializes state for the producer GenServer.

Starts an OffBroadway.Kafka producer process linked to the current process.

Handles termination of the Elsa consumer group supervisor process if one exists.

Link to this section Functions

Link to this function

assignments_revoked(pid)

View Source
@spec assignments_revoked(pid()) :: :ok

Passes Kafka partition assignment changes to the handle_info/2 function.

Called by Elsa assignments_revoked_handler.

Link to this function

handle_demand(demand, state)

View Source

Handles message events based on demand.

Updates the demand based on the existing demand in state and sends the requested number of message events.

Handles assignments revoked by the Kafka broker.

Waits until acknowledgers report as empty and cleans out any events in queue.

Link to this function

handle_messages(pid, messages)

View Source
@spec handle_messages(pid(), term()) :: :ok

Passes Kafka messages to the handle_info/2 function.

Called by Elsa message handler.

Names the producer process and initializes state for the producer GenServer.

If args contains a value for :endpoints, creates a handler config and passes it to the Elsa library to start a consumer group supervisor and stores the returned pid in the Broadway Producer state for reference.

Starts an OffBroadway.Kafka producer process linked to the current process.

Handles termination of the Elsa consumer group supervisor process if one exists.