View Source Poolex (poolex v0.2.2)

Poolex is a library for managing a pool of processes. Inspired by poolboy.

requirements

Requirements

RequirementVersion
Erlang/OTP>= 22
Elixir>= 1.7

installation

Installation

Add :poolex to your list of dependencies in mix.exs:

def deps do
  [
    {:poolex, "~> 0.2.0"}
  ]
end

usage

Usage

This example is based on the Elixir School's poolboy guide.
You can find the source of the below example here: poolex_example.

defining-the-worker

Defining the worker

We describe an actor that can easily become a bottleneck in our application, since it has a rather long execution time on a blocking call.

defmodule PoolexExample.Worker do
  use GenServer

  def start do
    GenServer.start(__MODULE__, nil)
  end

  def init(_args) do
    {:ok, nil}
  end

  def handle_call({:square_root, x}, _from, state) do
    IO.puts("process #{inspect(self())} calculating square root of #{x}")
    Process.sleep(1_000)
    {:reply, :math.sqrt(x), state}
  end
end

configuring-poolex

Configuring Poolex

defmodule PoolexExample.Application do
  @moduledoc false

  use Application

  defp worker_config do
    [
      worker_module: PoolexExample.Worker,
      workers_count: 5
    ]
  end

  def start(_type, _args) do
    children = [
      %{
        id: :worker_pool,
        start: {Poolex, :start_link, [:worker_pool, worker_config()]}
      }
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

List of possible configuration options:

OptionDescriptionExampleDefault value
worker_moduleName of module that implements our workerMyApp.Workeroption is required
worker_start_funName of the function that starts the worker:run:start
worker_argsList of arguments passed to the start function[:gg, "wp"][]
workers_countHow many workers should be running in the pool5option is required

using-poolex

Using Poolex

Poolex.run/3 is the function that you can use to interface with the worker pool.

  • The first parameter is the pool ID (see Poolex configuration).
  • The second parameter is a function that takes the pid of the worker and performs the necessary operation with it.
  • The third parameter is a keyword of run options.
    • :timeout -- Worker timeout on the side of the calling process. For example, if the timeout is 1000 and no free workers have appeared in the pool for a second, then the execution will abort with raising an error. The default value for this parameter is 5 seconds.
defmodule PoolexExample.Test do
  @timeout 60_000

  def start do
    1..20
    |> Enum.map(fn i -> async_call_square_root(i) end)
    |> Enum.each(fn task -> await_and_inspect(task) end)
  end

  defp async_call_square_root(i) do
    Task.async(fn ->
      Poolex.run!(
        :worker_pool,
        fn pid ->
          # Let's wrap the genserver call in a try - catch block. This allows us to trap any exceptions
          # that might be thrown and return the worker back to Poolex in a clean manner. It also allows
          # the programmer to retrieve the error and potentially fix it.
          try do
            GenServer.call(pid, {:square_root, i})
          catch
            e, r ->
              IO.inspect("Poolex transaction caught error: #{inspect(e)}, #{inspect(r)}")
              :ok
          end
        end,
        timeout: @timeout
      )
    end)
  end

  defp await_and_inspect(task), do: task |> Task.await(@timeout) |> IO.inspect()
end

Run the test function PoolexExample.Test.start() and see the result:

process #PID<0.227.0> calculating square root of 5
process #PID<0.223.0> calculating square root of 1
process #PID<0.225.0> calculating square root of 3
process #PID<0.224.0> calculating square root of 2
process #PID<0.226.0> calculating square root of 4
{:ok, 1.0}
{:ok, 1.4142135623730951}
{:ok, 1.7320508075688772}
{:ok, 2.0}
{:ok, 2.23606797749979}
{:ok, 2.449489742783178}
...

Link to this section Summary

Functions

Returns a specification to start this module under a supervisor.

Returns current state of started pool.

Same as run!/3 but handles runtime_errors.

The main function for working with the pool.

Starts a Poolex process without links (outside of a supervision tree).

Starts a Poolex process linked to the current process.

Link to this section Types

@type pool_id() :: atom()
@type poolex_option() ::
  {:worker_module, module()}
  | {:worker_start_fun, atom()}
  | {:worker_args, [any()]}
  | {:workers_count, pos_integer()}
@type run_option() :: {:timeout, timeout()}

Link to this section Functions

Returns a specification to start this module under a supervisor.

See Supervisor.

@spec get_state(pool_id()) :: Poolex.State.t()

Returns current state of started pool.

Primarily needed to help with debugging.

examples

Examples

iex> Poolex.start(:my_pool, worker_module: Agent, worker_args: [fn -> 0 end], workers_count: 5)
iex> state = %Poolex.State{} = Poolex.get_state(:my_pool)
iex> state.busy_workers_count
0
iex> state.idle_workers_count
5
iex> state.worker_module
Agent
Link to this function

run(pool_id, fun, options \\ [])

View Source
@spec run(pool_id(), (worker :: pid() -> any()), [run_option()]) ::
  {:ok, any()} | :all_workers_are_busy | {:runtime_error, any()}

Same as run!/3 but handles runtime_errors.

Returns:

  • {:runtime_error, reason} on errors.
  • :all_workers_are_busy if no free worker was found before the timeout.

See run!/3 for more information.

examples

Examples

iex> Poolex.start_link(:some_pool, worker_module: Agent, worker_args: [fn -> 5 end], workers_count: 1)
iex> Poolex.run(:some_pool, fn _pid -> raise RuntimeError end)
{:runtime_error, %RuntimeError{message: "runtime error"}}
iex> Poolex.run(:some_pool, fn pid -> Agent.get(pid, &(&1)) end)
{:ok, 5}
Link to this function

run!(pool_id, fun, options \\ [])

View Source
@spec run!(pool_id(), (worker :: pid() -> any()), [run_option()]) :: any()

The main function for working with the pool.

When executed, an attempt is made to obtain a worker with the specified timeout (5 seconds by default). In case of successful execution of the passed function, the result will be returned, otherwise an error will be raised.

examples

Examples

iex> Poolex.start_link(:some_pool, worker_module: Agent, worker_args: [fn -> 5 end], workers_count: 1)
iex> Poolex.run!(:some_pool, fn pid -> Agent.get(pid, &(&1)) end)
5
@spec start(pool_id(), [poolex_option()]) :: GenServer.on_start()

Starts a Poolex process without links (outside of a supervision tree).

See start_link/2 for more information.

examples

Examples

iex> Poolex.start(:my_pool, worker_module: Agent, worker_args: [fn -> 0 end], workers_count: 5)
iex> %Poolex.State{idle_workers_count: idle_workers_count} = Poolex.get_state(:my_pool)
iex> idle_workers_count
5
Link to this function

start_link(pool_id, opts)

View Source
@spec start_link(pool_id(), [poolex_option()]) :: GenServer.on_start()

Starts a Poolex process linked to the current process.

This is often used to start the Poolex as part of a supervision tree.

After the process is started, you can access it using the previously specified pool_id.

options

Options

OptionDescriptionExampleDefault value
worker_moduleName of module that implements our workerMyApp.Workeroption is required
worker_start_funName of the function that starts the worker:run:start
worker_argsList of arguments passed to the start function[:gg, "wp"][]
workers_countHow many workers should be running in the pool5option is required

examples

Examples

iex> Poolex.start_link(:my_pool, worker_module: Agent, worker_args: [fn -> 0 end], workers_count: 5)
iex> %Poolex.State{idle_workers_count: idle_workers_count} = Poolex.get_state(:my_pool)
iex> idle_workers_count
5