Stargate.Producer (stargate v0.1.1) View Source

Provides a producer websocket process and functions for producing messages to the cluster.

Pass a keyword list of configuration options to the start_link/1 function or simply call produce/2 passing a valid Pulsar producer URL in place of a producer process.

Link to this section Summary

Types

Pulsar messages produced by Stargate can be any of the following forms

A producer websocket process identified by a pid or via tuple. The atom key for identifying the producer in the via tuple is of the form :"sg_prod_<tenant>_<namespace>_<topic>.

A URL defining the host and topic to which a Stargate producer can connect for sending messages.

Functions

Generate puid

Puid.Info module info

Produce a message or list of messages to the cluster by producer URL or producer process. Messages can be any of the accepted forms (see message type).

Produce a list of messages to a Stargate producer process. Messages can be any of the accepted forms (see message type).

Start a producer websocket process and link it to the current process.

Link to this section Types

Specs

message() ::
  String.t()
  | {String.t(), String.t()}
  | %{
      :payload => String.t(),
      optional(:key) => String.t(),
      optional(:context) => String.t(),
      optional(:properties) => map(),
      optional(:replicationClusters) => [String.t()]
    }

Pulsar messages produced by Stargate can be any of the following forms:

* raw binary payload (must be encodable to base64)
* a {key, value} tuple where key is the optional message key and value is the payload
* a map with a "payload" field and optional fields for a key, context, properties (key/value
  pairs as a map), and list of strings identifying replication clusters.

Stargate uses the context field on a message produced to Pulsar to correlate receipt messages from the cluster to sent messages. If you do not define a context in your message, Stargate generates one automatically.

Specs

producer() :: GenServer.server()

A producer websocket process identified by a pid or via tuple. The atom key for identifying the producer in the via tuple is of the form :"sg_prod_<tenant>_<namespace>_<topic>.

Specs

url() :: String.t()

A URL defining the host and topic to which a Stargate producer can connect for sending messages.

Link to this section Functions

Generate puid

Puid.Info module info

Specs

produce(url() | producer(), message() | [message()]) :: :ok | {:error, term()}

Produce a message or list of messages to the cluster by producer URL or producer process. Messages can be any of the accepted forms (see message type).

Producing by URL is good for irregular and/or ad hoc producer needs that do not require a persistent websocket connection and ideally with few to no query parameters to configure producer options from the default. For higher volume producing, a persistent connection with an addressable producer process is recommended.

Once the message(s) is produced, the calling process automatically blocks until it receives acknowledgement from the cluster that the message(s) has been received.

Link to this function

produce(producer, messages, mfa)

View Source

Specs

produce(producer(), message() | [message()], {module(), atom(), [term()]}) ::
  :ok | {:error, term()}

Produce a list of messages to a Stargate producer process. Messages can be any of the accepted forms (see message type).

When calling produce/3 the third argument must be an MFA tuple which is used by the producer's acknowledger process to asynchronously perform acknowledgement that the message was received by the cluster successfully. This is used to avoid blocking the calling process for performance reasons.

Specs

start_link(keyword()) :: GenServer.on_start()

Start a producer websocket process and link it to the current process.

Producer options require, at minimum:

* `host` is a tuple of the address or URL of the Pulsar cluster (broker service)
  and the port on which the service is exposed.
* `tenant` is a string representing the tenant portion of the producer URL path parameter.
* `namespace` is a string representing the namespace portion of the producer URL path parameter.
* `topic` is a string representing the topic portion of the producer URL path parameter.
* `registry` is the name of the process registry associated to the client's supervision tree.
  Stargate uses this to send messages back and forth between the producer and its acknowledger.

Additional optional parameters to a producer are:

* `protocol` can be one of "ws" or "wss"; defaults to "ws"
* `persistence` can be one of "persistent" or "non-persistent" per the Pulsar
  specification of topics as being in-memory only or persisted to the brokers' disks.
  Defaults to "persistent".
* `query_params` is a map containing any or all of the following:

    * `send_timeout` the time at which a produce operation will time out; defaults to 30 seconds
    * `batch_enabled` can be true or false to enable/disable the batching of messages.
      Defaults to "false".
    * `batch_max_msg` defines the maximum number of messages in a batch (if enabled).
      Defaults to 1000.
    * `max_pending_msg` defines the maximum size of the internal queue holding messages. Defaults
      to 1000.
    * `batch_max_delay` sets the time period within which message batches will be published.
      Defaults to 10 milliseconds.
    * `routing_mode` can be one of :round_robin or :single. _Pulsar has deprecated this parameter_.
    * `compression_type` can be one of :lz4, :zlib, or :none. Defaults to :none
    * `name` is used to enforce only one producer with the given name is publishing to
      connected topic.
    * `initial_seq_id` sets the baseline for the sequence ids assigned to published messages.
    * `hashing_scheme` can be one of :java_string or :murmur3 when defining a hashing function to
      use with partitioned topics. _Pulsar has deprecated this parameter_.