View Source Consumer Example

Ensure :brod is added to your deps on mix.exs

defp deps do
    [
      {:brod, "~> 3.10.0"}
    ]
end

consumer

Consumer

Either the brod_group_subscriber_v2 or brod_group_subscriber behaviours can be used to consume messages. The key difference is that the v2 subscriber runs a worker for each partition in a separate Erlang process, allowing parallel message processing.

Here is an example of callback module that implements the brod_group_subscriber_v2 behaviour to consume messages.

defmodule BrodSample.GroupSubscriberV2 do
  @behaviour :brod_group_subscriber_v2

  def child_spec(_arg) do
    config = %{
      client: :kafka_client,
      group_id: "consumer_group_name",
      topics: ["streaming.events"],
      cb_module: __MODULE__,
      consumer_config: [{:begin_offset, :earliest}],
      init_data: [],
      message_type: :message_set,
      group_config: [
        offset_commit_policy: :commit_to_kafka_v2,
        offset_commit_interval_seconds: 5,
        rejoin_delay_seconds: 60,
        reconnect_cool_down_seconds: 60
      ]
    }
    
    %{
      id: __MODULE__,
      start: {brod_group_subscriber_v2, :start_link, [config]},
      type: :worker,
      restart: :temporary,
      shutdown: 5000
    }
  end
  
  @impl :brod_group_subscriber_v2
  def init(_group_id, _init_data), do: {:ok, []}

  @impl :brod_group_subscriber_v2
  def handle_message(message, _state) do
    IO.inspect(message, label: "message")
    {:ok, :commit, []}
  end
end

The example module implements child_spec/1 so that our consumer can be started by a Supervisor. The restart policy is set to :temporary because, in this case, if a message can not be processed, then there is no point in restarting. This might not always be the case.

See brod_group_subscriber_v2:start_link/1 for details on the configuration options.

See docs for more details about the required or optional callbacks.