ImagePipe.ProcessingPool (image_pipe v0.1.0)

Copy Markdown View Source

Limits how many images are processed at once, with a queue for requests that wait for a turn and a deadline for each image.

children = [
  {ImagePipe.ProcessingPool, name: MyApp.Pool, max_concurrency: 8, max_queue: 16},
  {ImagePipe, name: MyApp.Images, processing_pool: MyApp.Pool, sources: [...]},
  MyAppWeb.Endpoint
]

Every configuration that names the pool with :processing_pool shares its capacity, across mounts and ImagePipe.run/4 calls on the same node. Workers finishing after a timeout or cancellation keep their slots until their operations and cleanup finish. Limiting concurrent processing covers choosing the limits and what counts toward them.

Errors

ImagePipe.run/4 returns these errors, and a request gets a 503 for each:

  • {:processing, :overloaded} - the queue is full.
  • {:processing, :queue_timeout} - the request waited longer than :queue_timeout.
  • {:processing, :timeout} - the image took longer than :processing_timeout.
  • {:processing, :unavailable} - the pool isn't running.

Options

  • :name (atom/0) - Name that ImagePipe.config/1's :processing_pool refers to.

  • :max_concurrency (pos_integer/0) - Required. Most images processed at once.

  • :max_queue (non_neg_integer/0) - Most requests waiting for a turn. A request that finds the queue full fails with {:processing, :overloaded}. With 0, requests never wait. The default value is 64.

  • :queue_timeout (pos_integer/0) - Longest wait for a turn, in milliseconds. A request that waits longer fails with {:processing, :queue_timeout}. The default value is 10000.

  • :processing_timeout (pos_integer/0) - Request deadline once processing starts, in milliseconds, from reading the original to sending the last byte. A request that takes longer fails with {:processing, :timeout}. Its worker keeps the slot until the current operation and cleanup finish. The default value is 30000.

Summary

Functions

Returns a specification to start this module under a supervisor.

Starts a pool with validated admission and timeout options.

Returns the number of workers holding processing slots and of requests waiting, as %{active: 3, queued: 0}.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

start_link(opts)

Starts a pool with validated admission and timeout options.

stats(pool)

Returns the number of workers holding processing slots and of requests waiting, as %{active: 3, queued: 0}.

Active workers include those finishing after a timeout or cancellation.