BroadwaySQSInit (broadway_sqs_init v0.1.0)

View Source

A utility module to ensure an SQS queue exists before starting a Broadway pipeline. It checks if the queue exists using ExAws.SQS.get_queue_url/1 and creates it with ExAws.SQS.create_queue/2 if it doesn't. Returns the queue URL for use in Broadway configuration.

Summary

Functions

Ensures the specified SQS queue exists, creating it if necessary.

Functions

ensure_queue_exists(queue_name, queue_attributes, config_overrides \\ [])

@spec ensure_queue_exists(
  queue_name :: String.t(),
  queue_attributes :: map(),
  config_overrides :: keyword()
) :: String.t()

Ensures the specified SQS queue exists, creating it if necessary.

Parameters

  • opts: A map containing:
    • :queue_name (required): The name of the SQS queue.
    • :queue_attributes (optional): A map of queue attributes (e.g., fifo_queue: true).
    • :aws_config (required): A keyword list with AWS credentials (access_key_id, secret_access_key, region).

Returns

  • The queue URL as a string.

Raises

  • If the queue cannot be accessed or created, an exception is thrown with the error details.

Example

queue_url = BroadwaySQSInit.ensure_queue_exists(
  "my_queue.fifo",
  %{fifo_queue: true, content_based_deduplication: true},
  [
    access_key_id: System.get_env("AWS_ACCESS_KEY_ID"),
    secret_access_key: System.get_env("AWS_SECRET_ACCESS_KEY"),
    region: "us-east-1"
  ]
)

Broadway.start_link(MyBroadway,
  name: MyBroadway,
  producer: [
    module: {BroadwaySQS.Producer,
      queue_url: queue_url,
      config: [
        access_key_id: System.get_env("AWS_ACCESS_KEY_ID"),
        secret_access_key: System.get_env("AWS_SECRET_ACCESS_KEY"),
        region: "us-east-1"
      ]
    }
  ]
)