Kaffe v0.5.0 Kaffe.Producer

A GenServer for producing messages to a given Kafka topic.

Summary

Functions

Sync produce the key/value to the given topic and partition

Synchronously produce the given key/value to the first Kafka topic

Synchronously produce the key/value to topic

Synchronously produce the given key/value to the topic/partition

Start a Kafka producer

Functions

handle_call(msg, from, state)

Sync produce the key/value to the given topic and partition

produce_sync(key, value)

Synchronously produce the given key/value to the first Kafka topic.

This is a simpler way to produce if you’ve only given Producer a single topic for production and don’t want to specify the topic for each call.

Returns:

 * `:ok` on successfully producing the message
 * `{:error, reason}` for any error
produce_sync(topic, key, value)

Synchronously produce the key/value to topic

See produce_sync/2 for returns.

produce_sync(topic, partition, key, value)

Synchronously produce the given key/value to the topic/partition

See produce_sync/2 for returns.

start_link()

Start a Kafka producer

The producer pulls in values from the Kaffe producer configuration:

  • heroku_kafka_env - endpoints and SSL configuration will be pulled from ENV
  • endpoints - plaintext Kafka endpoints
  • topics - a list of Kafka topics to prep for producing
  • partition_strategy - the strategy to use when selecting the next partition. Default :md5.

    • :md5: provides even and deterministic distrbution of the messages over the available partitions based on an MD5 hash of the key
    • :round_robin - Cycle through the available partitions
    • :random - Select a random partition
    • function - Pass a function as an argument that accepts five arguments and returns the partition number to use for the message

      • topic, current_partition, partitions_count, key, value

On initialization the producer will analyze the given topic(s) and determine their available partitions. That analysis will be paired with the given partition selection strategy to select the partition per message.

Clients can also specify a partition directly when producing.

Currently only synchronous production is supported.