PulsarEx (pulsar_ex v0.1.1)

PulsarEx is a pure elixir/erlang client for Apache Pulsar. It's a wrapper of erlang client pulserl, with a bunch of bug fixes and improvements. This wrapper makes it easier to work with from within Elixir

Link to this section Summary

Functions

Synchronously produce message to pulsar, with default producer options

Being able to asynchronously send messages to pulsar is awesome, but being not able to catch the

errors sucks. So this sync version uses the async send, but wait for the batched/async sends to return

with result, pretty much like a future.

#

Starts the consumers

Examples

iex> defmodule ExampleCallback do
iex>   @behaviour PulsarEx.ConsumerCallback
iex>
iex>   def handle_messages(msgs, _state) do
iex>     msgs |> Enum.map(&{:ack, IO.inspect(&1)})
iex>   end
iex> end
iex>
iex> PulsarEx.start_consumers("test-topic", "test-subscription", ExampleCallback)
iex> PulsarEx.stop_consumers("test-topic", "test-subscription")

Available pulserl options:

consumer_name :: string()

priority_level :: non_neg_integer(),

Only works in shared/failover subscription

properties = [] :: keyword() | %{term() => term()}

initial_position = :latest :: :latest | :earliest

subscription_type = :shared :: :shared | :failover | :exclusive | :key_shared

queue_size = 1000 :: non_neg_integer()

acknowledgments_send_tick = 100 :: non_neg_integer()

max_pending_acknowledgments = 1000 :: non_neg_integer()

queue_refill_threshold :: non_neg_integer(),

When to get next batch of messages from broker, default to half of the queue_size

acknowledgment_timeout = 0 :: non_neg_integer(),

Timeout for sending acknowledgments, 0 means infinity.

If any, must be greater than 1_000(1 second)

nack_message_redelivery_delay = 60000 :: non_neg_integer(),

Redeliver delay for nack'ed messages

dead_letter_topic_name :: string()

dead_letter_topic_max_redeliver_count = 0 :: non_neg_integer(),

Max number of times nack'ed messages can be redelivered0 means infinity

Available PulsarEx options:

callback_module,

The module that implements PulsarEx.ConsumerCallback

batch_size = 1 :: non_neg_integer(),

Always consume the messages in batch, indicate the size of each batch, defaults to 1

poll_interval = 100,

How often to poll next batch, defaults to 100 milliseconds

workers = 1,

Number of workers to work on messages, default to 1.

Note in the case of consuming from partitioned topics and with key_shared subscription,

workers is set to be the same as number of partitions

See PulsarEx.Application.start_consumers/4.

See PulsarEx.Application.stop_consumers/2.

Link to this section Functions

Link to this function

produce(topic, payload, message_opts \\ [])

Synchronously produce message to pulsar, with default producer options

Being able to asynchronously send messages to pulsar is awesome, but being not able to catch the

errors sucks. So this sync version uses the async send, but wait for the batched/async sends to return

with result, pretty much like a future.

#

Example Producer Configs:

   config :pulserl,
     producer_opts: [
       properties: [test: true, env: staging],
       batch_enable: true
     ]

Examples

iex> PulsarEx.produce("test-topic", "test-payload", event_time: :os.system_time(:millisecond))

Default pulserl producer options are picked up from configs:

producer_name :: string()

properties = [] :: keyword() | %{term() => term()}

send_timeout = 30000 :: non_neg_integer()

Timeout sending messages to pulsar, defaults to 30 seconds.

Setting it to 0 means infinity

initial_sequence_id :: non_neg_integer()

batch_enable = true :: boolean()

routing_mode = :round_robin_routing :: :single_routing | :round_robin_routing

block_on_full_queue :: boolean()

batch_max_messages = 1000 :: non_neg_integer()

batch_max_delay_ms = 10 :: non_neg_integer()

max_pending_requests = 50000 :: non_neg_integer()

max_pending_requests_across_partitions = 100000 :: non_neg_integer()

Available PulsarEx message options:

timeout = 3000,

Timeout value for successfully send out the message to broker

partition_key, ordering_key :: string()

event_time :: non_neg_integer()

properties :: %{string() => string()} | [{string(), string()}]

deliver_at_time :: non_neg_integer(),

For delayed message delivery

Link to this function

start_consumers(topic, subscription, callback_module)

Starts the consumers

Examples

iex> defmodule ExampleCallback do
iex>   @behaviour PulsarEx.ConsumerCallback
iex>
iex>   def handle_messages(msgs, _state) do
iex>     msgs |> Enum.map(&{:ack, IO.inspect(&1)})
iex>   end
iex> end
iex>
iex> PulsarEx.start_consumers("test-topic", "test-subscription", ExampleCallback)
iex> PulsarEx.stop_consumers("test-topic", "test-subscription")

Available pulserl options:

consumer_name :: string()

priority_level :: non_neg_integer(),

Only works in shared/failover subscription

properties = [] :: keyword() | %{term() => term()}

initial_position = :latest :: :latest | :earliest

subscription_type = :shared :: :shared | :failover | :exclusive | :key_shared

queue_size = 1000 :: non_neg_integer()

acknowledgments_send_tick = 100 :: non_neg_integer()

max_pending_acknowledgments = 1000 :: non_neg_integer()

queue_refill_threshold :: non_neg_integer(),

When to get next batch of messages from broker, default to half of the queue_size

acknowledgment_timeout = 0 :: non_neg_integer(),

Timeout for sending acknowledgments, 0 means infinity.

If any, must be greater than 1_000(1 second)

nack_message_redelivery_delay = 60000 :: non_neg_integer(),

Redeliver delay for nack'ed messages

dead_letter_topic_name :: string()

dead_letter_topic_max_redeliver_count = 0 :: non_neg_integer(),

Max number of times nack'ed messages can be redelivered0 means infinity

Available PulsarEx options:

callback_module,

The module that implements PulsarEx.ConsumerCallback

batch_size = 1 :: non_neg_integer(),

Always consume the messages in batch, indicate the size of each batch, defaults to 1

poll_interval = 100,

How often to poll next batch, defaults to 100 milliseconds

workers = 1,

Number of workers to work on messages, default to 1.

Note in the case of consuming from partitioned topics and with key_shared subscription,

workers is set to be the same as number of partitions

Link to this function

start_consumers(topic, subscription, callback_module, opts)

See PulsarEx.Application.start_consumers/4.

Link to this function

stop_consumers(topic, subscription)

See PulsarEx.Application.stop_consumers/2.