View Source OffBroadway.Splunk.Producer (off_broadway_splunk v1.1.0)

GenStage Producer for a Splunk Event Stream. Broadway producer acts as a consumer for the specified Splunk SID.

producer-options

Producer Options

  • :sid - Required. The SID (Search ID) for the Splunk job we want to consume events from.

  • :receive_interval - The duration (in milliseconds) for which the producer waits before making a request for more messages. The default value is 5000.

  • :on_success - Configures the acking behaviour for successful messages. See the "Acknowledgements" section below for all the possible values. The default value is :ack.

  • :on_failure - Configures the acking behaviour for failed messages. See the "Acknowledgements" section below for all the possible values. The default value is :noop.

  • :splunk_client - A module that implements the OffBroadway.Splunk.Client behaviour. This module is responsible for fetching and acknowledging the messages from Splunk. All options passed to the producer will also be forwarded to the client. The default value is OffBroadway.Splunk.SplunkClient.

  • :config - A set of config options that overrides the default config for the splunk_client module. Any option set here can also be configured in config.exs. The default value is [].

    • :base_url - Base URL to Splunk instance.

    • :api_token - API token used to authenticate on the Splunk instance.

acknowledgements

Acknowledgements

You can use the on_success and on_failure options to control how messages are acknowledged. You can set these options when starting the Splunk producer or change them for each message through Broadway.Message.configure_ack/2. By default, successful messages are acked (:ack) and failed messages are not (:noop).

The possible values for :on_success and :on_failure are:

  • :ack - acknowledge the message. Splunk does not have any concept of acking messages, because we are just consuming messages from a web api endpoint. For now we are just executing a :telemetry event for acked messages.

  • :noop - do not acknowledge the message. No action are taken.

telemetry

Telemetry

This library exposes the following telemetry events:

  • [:off_broadway_splunk, :job_status, :start] - Dispatched before polling SID status from Splunk.

    • measurement: %{time: System.monotonic_time}
    • metadata: %{sid: string, progress: integer}
  • [:off_broadway_splunk, :job_status, :stop] - Dispatched when polling SID status from Splunk is complete.

    • measurement: %{time: native_time}
    • metadata: %{sid: string, progress: integer}
  • [:off_broadway_splunk, :job_status, :exception] - Dispatched after a failure while polling SID status from Splunk.

    • measurement: %{duration: native_time}

    • metadata:

      %{
        sid: string,
        kind: kind,
        reason: reason,
        stacktrace: stacktrace
      }
  • [:off_broadway_splunk, :receive_messages, :start] - Dispatched before receiving messages from Splunk (c:receive_messages/2)

    • measurement: %{time: System.monotonic_time}
    • metadata: %{sid: string, demand: integer}
  • [:off_broadway_splunk, :receive_messages, :stop] - Dispatched after messages have been received from Splunk and "wrapped".

    • measurement: %{time: native_time}

    • metadata:

      %{
        sid: string,
        received: integer,
        demand: integer
      }
  • [:off_broadway_splunk, :receive_messages, :exception] - Dispatched after a failure while receiving messages from Splunk.

    • measurement: %{duration: native_time}

    • metadata:

      %{
        sid: string,
        demand: integer,
        kind: kind,
        reason: reason,
        stacktrace: stacktrace
      }
  • [:off_broadway_splunk, :receive_messages, :ack] - Dispatched when acking a message.

    • measurement: %{time: System.system_time, count: 1}

    • meatadata:

      %{
        sid: string,
        receipt: receipt
      }