Kaffe v1.10.0 Kaffe.GroupMember

Note: The brod_group_member behavior is used.

Consume messages from a Kafka topic for a consumer group. There is one brod group member per topic! So as new topics are added to configuration so are the number of brod group members. Likewise, if you’re using something like Heroku Kafka and have multiple dynos for your consumer, there will be a Kaffe.GroupMember per dyno and each group member will receive a equal set of partition assignments for each topic.

The actual consumption is delegated to a series of subscribers, see Kaffe.Subscriber for details on how messages are processed.

The subscribers are assigned generations. Each generation represents a specific configuration. In order to allow the partitions to be rebalanced on startup, there is a delay between receiving a set of assignments associated with that generation and actually creating the subscribers. If a new generation is received in the mean time, the older generation is discarded.

See the follow for more details: https://github.com/klarna/brod/blob/master/src/brod_group_member.erl https://github.com/klarna/brucke/blob/master/src/brucke_member.erl

Summary

Functions

Callback implementation for c::brod_group_member.assign_partitions/3

Callback implementation for c::brod_group_member.assignments_received/4

Callback implementation for c::brod_group_member.assignments_revoked/1

Callback implementation for c::brod_group_member.get_committed_offsets/2

Functions

assign_partitions(pid, members, topic_partitions)

Callback implementation for c::brod_group_member.assign_partitions/3.

assignments_received(pid, member_id, generation_id, assignments)

Callback implementation for c::brod_group_member.assignments_received/4.

assignments_revoked(pid)

Callback implementation for c::brod_group_member.assignments_revoked/1.

get_committed_offsets(group_member_pid, topic_partitions)

Callback implementation for c::brod_group_member.get_committed_offsets/2.

start_link(subscriber_name, consumer_group, worker_manager_pid, topic)