Stargate (stargate v0.1.1) View Source

Stargate provides an Elixir client for the Apache Pulsar distributed message log service, based on the Pulsar project's websocket API.

Producer

Create a producer process under your application's supervision tree with the following:

options = [
    name: :pulsar_app,
    host: [{:"broker-url.com", 8080}],
    producer: [
        persistence: "non-persistent",
        tenant: "marketing",
        namespace: "public",
        topic: "new-stuff"
    ]
]

Stargate.Supervisor.start_link(options)

Once the producer is running, pass messages to the client by pid or by the named registry entry:

Stargate.produce(producer, [{"key", "value"}])

If you won't be producing frequently you can choose to run ad hoc produce commands against the url of the Pulsar cluster/topic as follows:

url = "ws://broker-url.com:8080/ws/v2/producer/non-persistent/marketing/public/new-stuff"

Stargate.produce(url, [{"key, "value"}])

Consumer and Reader

Both consumers and readers connected to Pulsar via Stargate process received messages the same way. Stargate takes care of receiving the messages and sending acknowledgements back to the cluster so all you need to do is start a process and define a module in your application that invokes use Stargate.Receiver.MessageHandler and has a handle_message/1 or handle_message/2 function as follows:

defmodule Publicize.MessageHandler do
    use Stargate.Receiver.MessageHandler

    def handle_message(%{context: context, payload: payload}) do
        publish_to_channel(payload, context)

        :ack
    end

    defp publish_to_channel(payload, context) do
        ...do stuff...
    end
end

The handle_message/1 must return either :ack or :continue in order to ack successful processing of the message back to the cluster or continue processing without ack (in the event you want to do a bulk/cumulative ack at a later time). If using the handle_message/2 callback for handlers that keep state across messages handled, it must return {:ack, state} or {:continue, state}.

Then, create a consumer or reader process under your application's supevision tree with the following:

options = [
    name: :pulsar_app,
    host: [{:"broker-url.com", 8080}]
    consumer: [                        <====== replace with `:reader` for a reader client
        tenant: "internal",
        namespace: "research",
        topic: "ready-to-release",
        subscription: "rss-feed",      <====== required for a `:consumer`
        handler: Publicizer.MessageHandler
    ]
]

Stargate.Supervisor.start_link(options)

Readers and Consumers share the same configuration API with the two key differences that the :consumer key in the options differentiates from the :reader key, as well as the requirement to provide a "subscription" to a consumer for the cluster to manage messages.

Link to this section Summary

Link to this section Functions