DurableServer.Terminator (durable_server v0.1.1)

Terminator GenServer that coordinates graceful shutdown of DurableServer processes.

The Terminator is placed at the bottom of the DurableServer.Supervisor supervision tree and traps exits. When the supervisor is shutting down, the Terminator's terminate/2 callback is called, which:

  1. Sends sync_and_stop messages to DurableServer children (with limited concurrency)
  2. Monitors each child process for DOWN messages
  3. Waits up to a configurable timeout for each child to sync and terminate
  4. Returns to continue the shutdown process

This ensures that DurableServer processes have an opportunity to persist their state before the supervisor tree is torn down, while preventing indefinite hangs during shutdown.

Configuration

The Terminator uses the same configuration as its parent DurableServer.Supervisor:

  • :graceful_shutdown_timeout_ms - Maximum time to wait for each child to shutdown (default: 30_000ms)
  • :graceful_shutdown_concurrency - Maximum concurrent shutdown operations (default: 50, should match Finch pool size to avoid connection exhaustion)

Graceful Shutdown Protocol

  1. Supervisor begins shutdown process
  2. Terminator's terminate/2 is called with reason and state
  3. Terminator uses Task.async_stream with limited concurrency to: a. Send {:durable, {:sync_and_stop, reason}} to each DurableServer b. Wait for each child to terminate (up to timeout)
  4. Each DurableServer calls sync_state/1 then stops normally
  5. After all children stop or timeout is reached, terminate/2 returns
  6. Supervisor continues shutdown process

The concurrency limit prevents overwhelming the Finch connection pool when many DurableServers try to persist their state simultaneously during shutdown.

The graceful shutdown only applies to normal shutdown scenarios (e.g., application stop, supervisor shutdown). For abnormal termination (crashes, kills), the normal supervision tree behavior applies.

Summary

Functions

Returns a specification to start this module under a supervisor.

Callback implementation for GenServer.init/1.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

init(arg)

Callback implementation for GenServer.init/1.

start_link(opts)