Stargate (stargate v0.2.0) View Source

Stargate provides an Elixir client for the Apache Pulsar distributed message log service, based on the Pulsar project's websocket API.

Producer

Create a producer process under your application's supervision tree with the following:

options = [
    name: :pulsar_app,
    host: [{:"broker-url.com", 8080}],
    producer: [
        persistence: "non-persistent",
        tenant: "marketing",
        namespace: "public",
        topic: "new-stuff"
    ]
]

Stargate.Supervisor.start_link(options)

Once the producer is running, pass messages to the client by pid or by the named registry entry:

Stargate.produce(producer, [{"key", "value"}])

If you won't be producing frequently you can choose to run ad hoc produce commands against the url of the Pulsar cluster/topic as follows:

url = "ws://broker-url.com:8080/ws/v2/producer/non-persistent/marketing/public/new-stuff"

Stargate.produce(url, [{"key, "value"}])

Consumer and Reader

Both consumers and readers connected to Pulsar via Stargate process received messages the same way. Stargate takes care of receiving the messages and sending acknowledgements back to the cluster so all you need to do is start a process and define a module in your application that invokes use Stargate.Receiver.MessageHandler and has a handle_message/1 or handle_message/2 function as follows:

defmodule Publicize.MessageHandler do
    use Stargate.Receiver.MessageHandler

    def handle_message(%{context: context, payload: payload}) do
        publish_to_channel(payload, context)

        :ack
    end

    defp publish_to_channel(payload, context) do
        ...do stuff...
    end
end

The handle_message/1 must return either :ack or :continue in order to ack successful processing of the message back to the cluster or continue processing without ack (in the event you want to do a bulk/cumulative ack at a later time). If using the handle_message/2 callback for handlers that keep state across messages handled, it must return {:ack, state} or {:continue, state}.

Then, create a consumer or reader process under your application's supevision tree with the following:

options = [
    name: :pulsar_app,
    host: [{:"broker-url.com", 8080}]
    consumer: [                        <====== replace with `:reader` for a reader client
        tenant: "internal",
        namespace: "research",
        topic: "ready-to-release",
        subscription: "rss-feed",      <====== required for a `:consumer`
        handler: Publicizer.MessageHandler
    ]
]

Stargate.Supervisor.start_link(options)

Readers and Consumers share the same configuration API with the two key differences that the :consumer key in the options differentiates from the :reader key, as well as the requirement to provide a "subscription" to a consumer for the cluster to manage messages.

Link to this section Summary

Functions

Generate the via-tuple needed for addressing a process within the Stargate supervision tree. Expects at minimum the tenant, namespace, and topic of the process being addressed and assumes by default the desired process is the Producer of a persistent topic managed by the default supervisor/registry.

Link to this section Types

Specs

component() ::
  :producer | :producer_ack | :consumer | :consumer_ack | :reader | :reader_ack

Specs

key_opt() ::
  {:persistence, persistence()}
  | {:name, atom()}
  | {:registry, atom()}
  | {:component, component()}

Specs

namespace() :: String.t()

Specs

persistence() :: String.t()

Specs

tenant() :: String.t()

Specs

topic() :: String.t()

Link to this section Functions

Link to this function

produce(url_or_connection, message)

View Source

See Stargate.Producer.produce/2.

Link to this function

produce(connection, message, mfa)

View Source

See Stargate.Producer.produce/3.

Link to this function

registry_key(tenant, namespace, topic, opts \\ [])

View Source

Specs

registry_key(tenant(), namespace(), topic(), [key_opt()]) ::
  {:via, Registry,
   {atom(), {component(), persistence(), tenant(), namespace(), topic()}}}

Generate the via-tuple needed for addressing a process within the Stargate supervision tree. Expects at minimum the tenant, namespace, and topic of the process being addressed and assumes by default the desired process is the Producer of a persistent topic managed by the default supervisor/registry.

iex> Stargate.registry_key("foo", "bar", "baz") {:via, Registry, {:sg_reg_default, {:producer, "persistent", "foo", "bar", "baz"}}}

iex> Stargate.registry_key("foo", "bar", "baz", registry: MyCustom.Registry, persistence: "non-persistent", component: :producer_ack) {:via, Registry, {MyCustom.Registry, {:producer_ack, "non-persistent", "foo", "bar", "baz"}}}