Neutron (neutron v0.1.0)

A C nif-based Apache Pulsar Client

Link to this section Summary

Functions

Does a asynchronous produce using the given producer pid generated from create_async_producer, given message binary and optional produce_opts. produce_opts: the keys are :deliver_after_ms and :deliver_at_ms both take an int as the value for :deliver_at_ms the int is a unix timestamp in milliseconds for :deliver_after_ms it's the delay to send the message after in milliseconds Uses the global pulsar client for connection information. Will return :ok on sucess or an {:error, String.t()} on failure

Creates an asynchronous producer genServer with associated topic and callback. Returns a pid to be managed and re-used by the callie.

Same args and options as create_async_producer but

Register and start Pulsar consumer under the Neutron supervisor tree

Does a synchronous produce to given topic binary with given message binary and optional produce_opts. produce_opts: the keys are :deliver_after_ms and :deliver_at_ms both take an int as the value for :deliver_at_ms the int is a unix timestamp in milliseconds for :deliver_after_ms it's the delay to send the message after in milliseconds. Will bootstrap a pulsar producer for each call so async produce where you re-use the producer is more efficient. Uses the global pulsar client for connection information.

Link to this section Functions

Link to this function

async_produce(producer_lookup, msg, produce_opts \\ %{})

Specs

async_produce(Genserver.server(), String.t(), map()) :: GenServer.call()

Does a asynchronous produce using the given producer pid generated from create_async_producer, given message binary and optional produce_opts. produce_opts: the keys are :deliver_after_ms and :deliver_at_ms both take an int as the value for :deliver_at_ms the int is a unix timestamp in milliseconds for :deliver_after_ms it's the delay to send the message after in milliseconds Uses the global pulsar client for connection information. Will return :ok on sucess or an {:error, String.t()} on failure

Link to this function

create_async_producer(topic, callback_module, genserver_opts \\ [])

Specs

create_async_producer(String.t(), atom(), GenServer.options()) ::
  GenServer.on_start()

Creates an asynchronous producer genServer with associated topic and callback. Returns a pid to be managed and re-used by the callie.

The callback function is applied to each message delivered from Pulsar to this producer. It needs to implement the PulsarAsyncProducerCallback behaviour. (@callback handle_delivery({:ok, String.t()} | {:error, String.t()}) :: any())

The callback function's return value is ignored. It is either passed in {:ok, String.t()} | {:error, String.t()} depending on if the message was successfully produced or if there was an error. Where the String.t() is the serialized messageId which is comma seperated surrounded by parentheses. This can be re-serialized and passed back to pulsar, if needed:

int64 ledgerId int64 entryId int32 partition int32 batchIndex

Link to this function

create_managed_async_producer(topic, callback_module, genserver_opts \\ [])

Specs

create_managed_async_producer(String.t(), atom(), GenServer.options()) ::
  DynamicSupervisor.on_start_child()

Same args and options as create_async_producer but

starts the async producer under the neutron supervisor tree with a dynamic supervisor.
Link to this function

start_consumer(args \\ [])

Specs

Register and start Pulsar consumer under the Neutron supervisor tree

Using a keyword list [callback_module: ModuleName] value.

The callback needs to implement the PulsarConsumerCallback behaviour (@callback handle_message(String.t()) :: :ack | :ack_all | :nack)

The callback handle_message function is applied to each message from Pulsar to this consumer. The callback handle_message function should return one of these three atoms:

:ack :ack_all :nack

:ack and :ack_all have relatively transparent counterparts in the Pulsar network API. However :nack does not, and should be used with great care (it is implemented client-side).

A callback return value of :nack will cause the Pulsar client to accumulate a data structure of message ids. This periodically sends requests to Pulsar with the ids. There is no logic in the consumer to prevent this data structure from growing without bound. If the valid messages are not forthcoming, the Pulsar client /should/ run out of memory. As far as I know, this unfortunate flaw is inherent in the current design of Pulsar.

Another argument in the keyword list is consumer type with the valid types of: :exclusive, :shared, :failover, or :key_shared They are passed with a [consumer_type: :shared] keyword list value. This defaults to :shared. Also :shared does not support :ack_all as expected due to the protocol.

The last keyword list arguments are topic and subscription binaries ("strings"): i.e. [topic: "my-topic", subscription: "my-subscription"]. The default topic is "my-topic" and the default subscription "my-subscription" fallbacks are included.

Link to this function

stop_consumers_by_subscription(subscription)

Specs

stop_consumers_by_subscription(String.t()) :: [
  DynamicSupervisor.terminate_child()
]
Link to this function

stop_consumers_by_topic(topic)

Specs

stop_consumers_by_topic(String.t()) :: [DynamicSupervisor.terminate_child()]
Link to this function

sync_produce(topic, message, produce_opts \\ %{})

Specs

sync_produce(String.t(), String.t(), map()) :: :ok | {:error, String.t()}

Does a synchronous produce to given topic binary with given message binary and optional produce_opts. produce_opts: the keys are :deliver_after_ms and :deliver_at_ms both take an int as the value for :deliver_at_ms the int is a unix timestamp in milliseconds for :deliver_after_ms it's the delay to send the message after in milliseconds. Will bootstrap a pulsar producer for each call so async produce where you re-use the producer is more efficient. Uses the global pulsar client for connection information.