Kaffe v1.2.1 Kaffe.WorkerManager

Manage topic/partition-to-worker assignments. Subscribers get workers from here.

This process manages the workers, while delegating to Kaffe.WorkerSupervisor to start each worker under supervision.

Workers are allocated based on a configured strategy (worker_allocation_strategy in the consumer config): either one worker per topic/partition, or one worker for a partition across topics.

The first strategy is useful for higher throughput and increased flexibility. The second is useful when mutliple input topics may have the same messages (identified by key) and those messages must be processsed sequentially.

The table of workers is stored in an ETS table, :kaffe_workers.

Summary

Functions

start_link(subscriber_name)
worker_for(pid, topic, partition)
worker_name(topic, partition)