RabbitMQPoolEx (RabbitMQ Pool Ex v1.0.0)
View SourceRabbitMQPoolEx
is an Elixir library that provides a robust and efficient connection pooling mechanism for RabbitMQ.
It leverages poolboy
to manage a pool of connections, ensuring high performance and reliability in message-driven
applications.
Features
- Connection Pooling: Efficiently manage multiple RabbitMQ connections to handle high-throughput messaging.
- Channel Management: Simplify the process of acquiring and releasing channels from the connection pool.
- Fault Tolerance: Automatically handle connection drops and retries, ensuring minimal disruption to message processing.
- Channel Reuse: Optionally reuse channels within a pool to optimize resource management.
- Configurable Pooling Strategy: Customize the size, overflow, and behavior of connection pools.
Installation
To integrate RabbitMQPoolEx
into your project, add the following to your mix.exs
dependencies:
defp deps do
[
{:rabbitmq_pool_ex, "~> 1.0.0"}
]
end
Then, fetch and install the dependencies by running:
mix deps.get
Configuration
RabbitMQPoolEx
requires defining connection pools and RabbitMQ settings. The configuration consists of:
:rabbitmq_config
(keyword list) – General RabbitMQ connection parameters.:connection_pools
(list) – A list of poolboy configurations, where each pool represents a connection to RabbitMQ.
Each pool configuration should include:
:name
(tuple) – A two element tuple containing the process registration scope and an unique name for the pool (e.g., {:local, :default_pool}).:worker_module
(module) – Defaults toRabbitMQPoolEx.Worker.RabbitMQConnection
, which manages pool connections and channels.:size
(integer) – Number of connection processes in the pool.:channels
(integer) – Number of channels managed within the pool.:reuse_channels?
(boolean) – Defaults tofalse
. Determines if channels should be reused instead of replaced after being used.:max_overflow
(integer) – Maximum number of extra workers allowed beyond the initial pool size.
Example Configuration:
To use RabbitMQPoolEx
, add the following to your application's supervision tree:
defmodule MyApp.Application do
@moduledoc false
@impl true
def start(_type, _args) do
children = [
{RabbitMQPoolEx.PoolSupervisor, get_pool_config()}
]
opts = [strategy: :one_for_one, name: MyApp.Supervisor]
Supervisor.start_link(children, opts)
end
defp get_pool_config do
[
rabbitmq_config: [host: "localhost", port: 5672],
connection_pools: [
[
name: {:local, :default_pool},
size: 5,
channels: 20,
reuse_channels?: true,
max_overflow: 2
]
]
]
end
end
Usage
Once configured, you can interact with RabbitMQ through the pooled connections. Here's how to publish a message:
RabbitMQPoolEx.with_channel(:rabbitmq_pool, fn
{:ok, channel} ->
AMQP.Basic.publish(channel, "exchange_name", "routing_key", "Hello, World!")
:ok
{:error, reason} ->
IO.puts("Failed to acquire channel", error: inspect(reason))
end)
In this example, with_channel/2
checks out a channel from the pool, executes the given function, and ensures the
channel is returned to the pool afterward.
For more advanced usage, such as setting up consumers or handling different exchange types, refer to the detailed documentation and examples provided in the library's repository.
Manually retrieving a connection
To manually retrieve a RabbitMQ connection from the pool:
{:ok, conn} = RabbitMQPoolEx.get_connection(:default_pool)
Manually checking out and checking in a channel
{:ok, channel} = RabbitMQPoolEx.checkout_channel(worker_pid)
# Perform operations...
RabbitMQPoolEx.checkin_channel(worker_pid, channel)
In the example above, checkout_channel/1
retrieves a RabbitMQ channel from the connection worker,
and checkin_channel/2
returns it to the pool when done.
License
RabbitMQPoolEx is released under the Apache 2.0 License.
Summary
Types
The function to be used with with_channel/2
Functions
Returns a RabbitMQ channel to its corresponding connection worker.
Retrieves a RabbitMQ channel from the specified connection worker.
Retrieves a RabbitMQ connection from a connection worker within the pool.
Retrieves a connection worker from the pool.
Executes a given function within the context of a RabbitMQ channel.
Types
@type client_function() :: ({:ok, AMQP.Channel.t()} | {:error, :disconnected | :out_of_channels} -> any())
The function to be used with with_channel/2
Functions
@spec checkin_channel(pid(), AMQP.Channel.t()) :: :ok
Returns a RabbitMQ channel to its corresponding connection worker.
Parameters
conn_worker
: PID of the connection worker.channel
: The RabbitMQ channel to be returned.
@spec checkout_channel(pid()) :: {:ok, AMQP.Channel.t()} | {:error, :disconnected | :out_of_channels}
Retrieves a RabbitMQ channel from the specified connection worker.
Parameters
conn_worker
: PID of the connection worker.
@spec get_connection(atom()) :: {:ok, AMQP.Connection.t()} | {:error, :disconnected}
Retrieves a RabbitMQ connection from a connection worker within the pool.
Parameters
pool_id
: Atom representing the pool identifier.
Retrieves a connection worker from the pool.
This function uses the pool for load distribution and does not isolate access to individual workers. The pool configuration strategy is FIFO.
Parameters
pool_id:
Atom representing the pool identifier.
@spec with_channel(atom(), client_function()) :: any()
Executes a given function within the context of a RabbitMQ channel.
This function:
- Retrieves the reference (pid) of connection worker from the pool and immediately put it back into the pool, so any other concurrent client can have access to it.
- Obtains a RabbitMQ channel from that worker.
- Executes the provided function using the channel.
- Returns the channel to the worker's pool.
Parameters
pool_id
: Atom representing the pool identifier.fun
: Function to be executed within the channel's context.