View Source Pgmq (Pgmq v0.2.1)

Thin wrapper over the pgmq extension

Provides APIs for sending, reading, archiving and deleting messages.

Use-macros

You can use Pgmq for the convenience of having a standardized repo and less convoluted function calls. By defining:

  # lib/my_app/pgmq.ex
  defmodule MyApp.Pgmq do
    use Pgmq, repo: MyApp.Repo
  end

You can then call MyApp.Pgmq.send_message("myqueue", "hello"), without passing in the MyApp.Repo

Summary

Types

Queue name

An Ecto repository

Functions

Archives list of messages, removing them from the queue and putting them into the archive

Creates a queue in the database

Deletes a batch of messages, removing them from the queue

Deletes a queue from the database

Returns metrics for a single queue

Returns a list of queues with stats

Returns a list of queues

Reads a message and instantly deletes it from the queue

Reads a batch of messages from a queue

Reads a batch of messages from a queue, but waits if no messages are available

Sends one message to a queue

Sets the visibility timeout of a message for X seconds from now

Types

@type queue() :: String.t()

Queue name

@type repo() :: Ecto.Repo.t()

An Ecto repository

Functions

Link to this function

archive_messages(repo, queue, messages)

View Source
@spec archive_messages(
  repo(),
  queue(),
  [message_id :: integer()] | [message :: Pgmq.Message.t()]
) ::
  :ok

Archives list of messages, removing them from the queue and putting them into the archive

This function can receive a list of either Message.t() or message ids. Mixed lists aren't allowed.

Link to this function

create_queue(repo, queue, opts \\ [])

View Source
@spec create_queue(repo(), queue(), opts :: Keyword.t()) :: :ok | {:error, atom()}

Creates a queue in the database

Notice that the queue name must:

  • have less than 48 characters
  • start with a letter
  • have only letters, numbers, and _

Accepts the following options:

  • :unlogged: Boolean indicating if the queue should be unlogged. Unlogged queues are faster to write to, but data may be lost in database crashes or unclean exits. Can't be used together with :partitioned.
  • :partitioned: indicates if the queue is partitioned. Defaults to false. Requires pg_partman extension.
  • :partition_interval: interval to partition the queue, required if :partitioned is true.
  • :retention_interval: interval for partition retention, required if :partitioned is true.
Link to this function

delete_messages(repo, queue, messages)

View Source
@spec delete_messages(repo(), queue(), [message_id :: integer()] | [Pgmq.Message.t()]) ::
  :ok

Deletes a batch of messages, removing them from the queue

This function can receive a list of either Message.t() or message ids. Mixed lists aren't allowed.

@spec drop_queue(repo(), queue()) :: :ok | {:error, atom()}

Deletes a queue from the database

Link to this function

get_metrics(repo, queue)

View Source
@spec get_metrics(repo(), queue()) :: [
  %{
    queue_name: String.t(),
    queue_length: pos_integer(),
    newest_msg_age_sec: pos_integer() | nil,
    oldest_msg_age_sec: pos_integer() | nil,
    total_messages: pos_integer(),
    scrape_time: DateTime.t()
  }
]

Returns metrics for a single queue

@spec get_metrics_all(repo()) :: [
  %{
    queue_name: String.t(),
    queue_length: pos_integer(),
    newest_msg_age_sec: pos_integer() | nil,
    oldest_msg_age_sec: pos_integer() | nil,
    total_messages: pos_integer(),
    scrape_time: DateTime.t()
  }
]

Returns a list of queues with stats

@spec list_queues(repo()) :: [
  %{
    queue_name: String.t(),
    is_partitioned: boolean(),
    is_unlogged: boolean(),
    created_at: DateTime.t()
  }
]

Returns a list of queues

Link to this function

pop_message(repo, queue)

View Source
@spec pop_message(repo(), queue()) :: Pgmq.Message.t() | nil

Reads a message and instantly deletes it from the queue

If there are no messages in the queue, returns nil.

Link to this function

read_message(repo, queue, visibility_timeout_seconds)

View Source
@spec read_message(repo(), queue(), visibility_timeout_seconds :: integer()) ::
  Pgmq.Message.t() | nil

Reads one message from a queue

Returns immediately. If there are no messages in the queue, returns nil.

Messages read through this function are guaranteed not to be read by other calls for visibility_timeout_seconds.

Link to this function

read_messages(repo, queue, visibility_timeout_seconds, count)

View Source
@spec read_messages(
  repo(),
  queue(),
  visibility_timeout_seconds :: integer(),
  count :: integer()
) :: [
  Pgmq.Message.t()
]

Reads a batch of messages from a queue

Messages read through this function are guaranteed not to be read by other calls for visibility_timeout_seconds.

Link to this function

read_messages_with_poll(repo, queue, visibility_timeout_seconds, count, opts \\ [])

View Source
@spec read_messages_with_poll(
  repo(),
  queue(),
  visibility_timeout_seconds :: integer(),
  count :: integer(),
  opts :: Keyword.t()
) :: [Pgmq.Message.t()]

Reads a batch of messages from a queue, but waits if no messages are available

Accepts two options:

  • :max_poll_seconds: the maximum duration of the poll. Defaults to 5.
  • :poll_interval_ms: dictates how often the poll is made database side. Defaults to 250. Can be tuned for lower latency or less database load, depending on your needs.

When there are messages available in the queue, returns immediately. Otherwise, blocks until at least one message is available, or max_poll_seconds is reached.

Notice that this function may put significant burden on the connection pool, as it may hold the connection for several seconds if there's no activity in the queue.

Messages read through this function are guaranteed not to be read by other calls for visibility_timeout_seconds.

Link to this function

send_message(repo, queue, encoded_message)

View Source
@spec send_message(repo(), queue(), encoded_message :: binary()) ::
  {:ok, Pgmq.Message.t()} | {:error, term()}

Sends one message to a queue

Link to this function

set_message_vt(repo, queue, message_id, visibility_timeout)

View Source
@spec set_message_vt(
  repo(),
  queue(),
  Pgmq.Message.t() | integer(),
  visibility_timeout :: integer()
) ::
  :ok

Sets the visibility timeout of a message for X seconds from now