Limiting concurrent processing

Copy Markdown View Source

A processing pool limits how many images ImagePipe processes at once. Requests beyond the limit wait in a queue or get a 503, and each image gets a deadline. Without a pool, every request starts processing right away, so a burst of requests can use all the CPU and memory. image_pipe_server always runs one.

This guide assumes ImagePipe is running in your app (see Getting started with Phoenix) or as image_pipe_server (see Getting started with the server).

Adding a pool

Plug

Start an ImagePipe.ProcessingPool before the instance, and name it in the instance's processing_pool:

# lib/my_app/application.ex
children = [
  {ImagePipe.ProcessingPool, name: MyApp.Pool, max_concurrency: 8, max_queue: 16},
  {ImagePipe, name: MyApp.Images, processing_pool: MyApp.Pool, sources: [...]},
  MyAppWeb.Endpoint
]

Calls to ImagePipe.run/4 with the instance's name use the same pool, so jobs and requests share its capacity.

image_pipe_server

The server always runs a pool. Change its limits in a [pool] section of config.toml:

[pool]
max_concurrency = 8
max_queue = 16

Choosing the limits

  • max_concurrency is how many images are processed at once. Start with about the number of CPU cores, which is the server's default.
  • max_queue is how many requests may wait for a turn, 64 by default. A request that finds the queue full gets a 503. With 0, every request beyond max_concurrency gets a 503 straight away.
  • queue_timeout, 10 seconds by default, is how long a request waits for a turn before it gets a 503.
  • processing_timeout, 30 seconds by default, is the request deadline once processing starts.

ImagePipe.ProcessingPool lists the options, and the server reference their TOML keys.

What uses the pool

Every image ImagePipe processes takes a turn: resized images, info, BlurHash, and LQIP placeholders, whether a request or a call to ImagePipe.run/4 asked for it. These don't:

  • Images served from the cache.
  • 304 Not Modified responses.
  • Requests waiting for an identical request that is already being processed (see request coalescing).

Deadlines

The processing_timeout starts when a request gets its turn, and covers reading the original, processing, and sending the last byte of the response. A client that reads the response slowly uses up the deadline too.

When the deadline passes before the response starts, the request gets a 503. When the response has already started, ImagePipe cuts it short, as described in failures during streaming. It also cuts the response short when producing its next chunk takes longer than 60 seconds, whatever the deadline.

After a timeout or client disconnect, the worker keeps its processing slot until its current operation and resource cleanup finish. A libvips operation already running may continue after the request ends. Waiting requests enter only when a slot is released, and a cancelled request in the queue leaves immediately. Keep the size limits as well.

Monitoring the pool

The pool emits [:processing, :admission] for each wait for a turn and [:processing, :execute] for each processed image. In Elixir, ImagePipe.ProcessingPool.stats/1 returns how many workers still hold slots, including workers finishing after a timeout, and how many requests are waiting. The processing execution span ends when the worker finishes. ImagePipe.ProcessingPool lists the errors ImagePipe.run/4 returns for each case that gives a 503 over HTTP.