View Source Offbroadway.EventRelay.Producer (offbroadway_eventrelay v0.2.0)
A GenStage producer that continuously receives messages from an EventRelay Destination/Topic
For a quick getting started on using Broadway with EventRelay, please see the EventRelay Guide.
Example
defmodule BE.Broadway do
use Broadway
alias Broadway.Message
def start_link(_opts) do
Broadway.start_link(__MODULE__,
name: __MODULE__,
producer: [
module:
{Offbroadway.EventRelay.Producer,
destination_id: "...",
host: "localhost",
port: "50051",
token: "...",
certfile: "/path/to/clientcertfile.pem",
keyfile: "/path/to/clientkeyfile.pem",
cacertfile: "/path/to/cacertfile.pem"},
],
processors: [
default: [concurrency: 10]
]
)
end
def handle_message(_processor_name, message, _context) do
# Do something with the message/event
message
end
end
The above configuration will set up a producer that continuously pulls events from the configured destination and sends them downstream.
Options
:destination_id
- Required. The ID of the destination:host
- Required. The host for EventRelay The default value is:noop
.:port
- Required. The port for the GRPC API for EventRelay The default value is:noop
.:token
- An ApiKey token that has access to the EventRelay destination:cacertfile
- Path to a PEM formatted CA certificate file:certfile
- Path to a PEM formatted client certificate file:keyfile
- Path to a PEM formatted client key file:pull_interval
(integer/0
) - The duration (in milliseconds) for which the producer waits before making a request for more messages. The default value is5000
.:pull_timeout
- The maximum time (in milliseconds) to wait for a response before the pull client returns an error. The default value is:infinity
.
Authentication
For specifics of how to setup authetication in EventRelay use the Authentication Guide in the EventRelay Wiki.
See the Options section for authentication configuration options.
Acknowledgements
This producer uses the PullQueuedEventsRequest
in EventRelay which locks
an event for a specific destination when it is pulled from the system.
That means any client that pulls events for that destination in the
future will not receive that event.
Dealing with failures it is up to the user of this producer to implement your own failure handling via your own acknowledgement handling. See the example below.
transformer: {__MODULE__, :transform, []}
in combination with
acknowledger: {__MODULE__, :ack_id, :ack_data}
is what allows you to
setup up your own acknowledger.
In the future EventRelay will support unlocking an event via the GRPC API which would allow you to re-enqueue an event for processing again on failure.
Example
defmodule BE.Broadway do
use Broadway
alias Broadway.Message
def start_link(_opts) do
Broadway.start_link(__MODULE__,
name: __MODULE__,
producer: [
module:
{Offbroadway.EventRelay.Producer,
destination_id: "...",
host: "localhost",
port: "50051",
token: "...",
certfile: "/path/to/clientcertfile.pem",
keyfile: "/path/to/clientkeyfile.pem",
cacertfile: "/path/to/cacertfile.pem"},
transformer: {__MODULE__, :transform, []}
],
processors: [
default: [concurrency: 10]
]
)
end
def transform(message, _opts) do
%{
message
| acknowledger: {__MODULE__, :ack_id, :ack_data}
}
end
def handle_message(_processor_name, message, _context) do
# Do something with the message/event
message
end
def ack(:ack_id, _successful, _failed) do
# Handle the failed messages/events here
end
end