WaitForIt (WaitForIt v2.2.1)

Copy Markdown View Source

Various ways of waiting for things to happen.

WaitForIt lets you wait on the results of asynchronous or remote operations using intuitive, familiar syntax built on Elixir's own control-flow constructs (if, case, cond, and with). It is equally at home coordinating concurrent processes in production code and taming flaky timing in tests.

# Wait until a record shows up, and bind it directly:
{:ok, user} = WaitForIt.match_wait({:ok, %User{}}, Repo.fetch(User, id), timeout: 2_000)

Elixir provides several language and standard library features — such as Process.sleep/1, receive/1/after, and Task.async/1/Task.await/2 — that can be used to implement waiting, but they are inconvenient for the purpose. WaitForIt builds on top of them to provide convenient, expressive facilities for waiting on specific conditions. This is most obviously useful in tests that must wait for concurrent or asynchronous activity to complete, but it is just as useful anywhere concurrent processes coordinate their activity — asynchronous event handling, producer-consumer processes, and time-based activity.

To use WaitForIt, require WaitForIt or import WaitForIt.

If you are just getting started, the task-focused guides walk through the most common scenarios: Waiting in tests, Polling vs signaling, Composing waits, Recipes, and Telemetry.

The five forms of waiting

FormWaits until…Looks like
wait/2an expression is truthya bare expression
match_wait/3an expression matches a pattern (binding out of it)a <- clause
case_wait/3an expression matches one of several clausesa case expression
cond_wait/2one of several expressions is truthya cond expression
with_wait/3several composed waits all succeeda with expression

Each form has a ! variant (wait!/2, match_wait!/3, …) that raises WaitForIt.TimeoutError on timeout instead of returning a falsy value or raising the matching built-in error.

wait

wait/2 waits until a given expression evaluates to a truthy value.

# Wait up to one minute for a file to exist, then print its contents.
if WaitForIt.wait(File.exists?("data.csv"), timeout: :timer.minutes(1)) do
  IO.puts(File.read!("data.csv"))
else
  IO.warn("Stopped waiting for the file to exist")
end

match_wait

match_wait/3 waits until a given expression matches a given pattern, and binds out of it. It is the most convenient form when waiting for a tagged result such as {:ok, value}.

{:ok, user} = WaitForIt.match_wait({:ok, %User{}}, Repo.fetch(User, id), timeout: 2_000)

case_wait

case_wait/3 waits until a given expression matches one of the given case clauses. It looks and acts like a case/2 expression, except that it can take an optional else clause.

WaitForIt.case_wait(File.stat("data"), timeout: :timer.seconds(30)) do
  {:ok, %File.Stat{type: :directory}} ->
    File.write!("data/greeting.txt", "Hello, world!")
else
  {:ok, %File.Stat{type: type}} ->
    IO.warn("Expected 'data' to be a directory but its type is #{inspect(type)}")

  {:error, reason} ->
    IO.warn("Could not stat 'data': #{inspect(reason)}")
end

cond_wait

cond_wait/2 waits until any one of the given expressions evaluates to a truthy value. It looks and acts like a cond/1 expression, except that it can take an optional else clause.

WaitForIt.cond_wait(timeout: :timer.seconds(10), interval: 500) do
  File.exists?("data/process.json") -> IO.puts("Processing...")
  NaiveDateTime.utc_now().second == 0 -> IO.puts("Top of the minute!")
else
  IO.warn("Stopped waiting since neither condition ever became truthy")
end

with_wait

with_wait/3 composes several waits in a pipeline. It looks and acts like a with/1 expression, except that its <~ clauses wait until their expression matches.

WaitForIt.with_wait on(
  {:ok, account} <~ {load_account(token), timeout: 2_000},
  {:ok, balance} <~ fetch_balance(account)
) do
  {:ok, balance}
else
  not_ready -> {:error, {:timed_out, not_ready}}
end

Options

All forms of waiting accept the same options:

OptionDefaultDescription
:timeout5_000total time to wait, in milliseconds, before giving up
:interval100polling interval, in milliseconds, between re-evaluations (alias: :frequency)
:pre_wait0delay before the first evaluation, in milliseconds
:signaldisable polling and re-evaluate only when the named signal is received

See the Polling-based waiting and Signal-based waiting sections below for the :interval and :signal options. The :interval option may also be a WaitForIt.Backoff function for exponential or custom backoff.

:frequency is now :interval

The :frequency option has been renamed to :interval, which more accurately describes a time value in milliseconds. :frequency continues to work as an alias and is slated for removal in a future major version. If both are given, :interval takes precedence.

Timeout behavior

The forms of waiting differ in what happens when a wait times out. This table summarizes the behavior; the non-bang forms mirror the corresponding built-in Elixir construct, while every bang form raises a WaitForIt.TimeoutError.

ConstructOn timeout (no else)On timeout (with else)Bang variant raises
wait/2returns the last falsy value(no else clause)TimeoutError
match_wait/3raises MatchError(no else clause)TimeoutError
case_wait/3raises CaseClauseErrorevaluates elseTimeoutError
cond_wait/2raises CondClauseErrorevaluates elseTimeoutError
with_wait/3returns the last valueevaluates else (a <~ timeout flows here)TimeoutError (<~ clauses)

Waitable expressions and waiting conditions

A waitable expression is any Elixir expression that can be evaluated one or more times to produce a value. A waiting condition decides, from that value, whether to keep waiting or to halt with a result. For wait/2 the waiting condition is implicit (the truthiness of the expression); for case_wait/3 and with_wait/3 it is the case clauses or <~ patterns; for cond_wait/2 it is the truthiness of each branch.

Because a waitable expression is re-evaluated until its waiting condition is met, idempotent expressions are of little use — they would either halt immediately or never halt. It is expected that the value may change on each re-evaluation, and that evaluation may have side effects. Any such side effects must be safe to repeat, since the expression may be evaluated an indeterminate number of times while waiting.

Polling-based waiting

By default, WaitForIt polls: it re-evaluates the waitable expression at a fixed interval until the waiting condition is met or the timeout elapses. The interval is controlled by the :interval option (default 100 ms; the legacy alias :frequency is also accepted).

The :interval option may also be a 1-arity function of the attempt number, which enables backoff strategies — for example, polling less aggressively as time goes on so as not to hammer a struggling dependency. See WaitForIt.Backoff for ready-made strategies such as exponential backoff with jitter.

Signal-based waiting

Signal-based waiting removes the polling loop: instead of re-checking on a timer, a waiter blocks until it receives a signal telling it to re-evaluate. Opt in with the :signal option, naming a signal (any term, typically an atom), and have the code that changes the condition call signal/1.

Imagine a producer-consumer problem in which a consumer waits for items to appear in a buffer while a separate producer occasionally places items in the buffer:

# CONSUMER process
WaitForIt.wait(Buffer.count() >= 4, signal: :buffer_filled)

# PRODUCER process — after putting some things in the buffer, signal waiters
Buffer.put(item)
WaitForIt.signal(:buffer_filled)

Both sides share the same signal name, which binds the producer to the consumer. A signal does not mean the condition is now satisfied — only that waiters should re-evaluate. The wait halts when its condition is met, or continues until the next signal or the timeout.

See the Polling vs signaling guide for guidance on choosing between the two modes.

Telemetry

Every wait emits :telemetry events under the [:wait_for_it, :wait] prefix — :start, :stop, and :exception — so you can observe how long waits take, how many evaluations they require, and how often they time out. The :stop event reports the wait duration, the number of evaluations, and whether the wait :matched or hit a :timeout.

See the Telemetry guide for the full measurement and metadata reference, plus examples of attaching handlers and wiring up Telemetry.Metrics.

Using WaitForIt in tests

Tests — especially integration and end-to-end tests — are one of the most common places to wait on asynchronous work. The WaitForIt.Test module provides ExUnit assertions (assert_eventually/2, refute_eventually/2, and assert_always/2) that wait and re-evaluate and, on timeout, fail with a regular ExUnit.AssertionError that includes the source expression and the last value seen:

defmodule MyApp.SomeTest do
  use ExUnit.Case
  use WaitForIt.Test

  test "the user is eventually confirmed" do
    assert_eventually {:ok, %User{confirmed: true}} = Repo.fetch(User, user_id)
  end
end

The waiting macros can also be used directly in tests when you want their exact return values or timeout semantics — wait/2, for example, returns its value and so drops straight into an assert. See the Waiting in tests guide for a full walkthrough.

A note on "catch-all" clauses

It is common to include "catch-all" clauses in normal case/2 and cond/1 expressions — a final _ clause, or a final always-truthy true condition. When using case_wait/3 and cond_wait/2, avoid such catch-all clauses: because they always match, they would halt the wait on the very first evaluation. Use an else clause instead, which is only evaluated on timeout and lets you customize the behavior and return value when a wait gives up.

Summary

wait

Wait until the given expression evaluates to a truthy value.

The same as wait/2 but raises a WaitForIt.TimeoutError exception if it fails.

match_wait

Wait until the given expression matches the given pattern.

case_wait

Wait until the given expression matches one of the case clauses in the given block.

The same as case_wait/3 but raises a WaitForIt.TimeoutError exception if it fails.

cond_wait

Wait until one of the expressions in the given block evaluates to a truthy value.

The same as cond_wait/2 but raises a WaitForIt.TimeoutError exception if it fails.

with_wait

Compose several waits in a with-style pipeline.

The same as with_wait/3 but a <~ clause that times out raises a WaitForIt.TimeoutError instead of routing to the else block.

signaling

Send a signal to indicate that any processes waiting on the signal should re-evaluate their waiting conditions.

Types

Type to represent an expression that can be waited on.

Options that can be used to control waiting behavior.

Options that can be used to control waiting behavior.

wait

wait(expression, opts \\ [])

(macro)

Wait until the given expression evaluates to a truthy value.

Returns the truthy value that ended the wait, or the last falsy value evaluated if a timeout occurred.

Warning

The value returned from this macro has changed as of version 2.0.

In previous versions, {:ok, value} would be returned for the success case, and {:timeout, timeout_milliseconds} would be returned for the timeout case.

As of version 2.0, the final value of the wait expression is returned directly, which will be a truthy value for the success case and a falsy value for the timeout case. This allows the wait/2 macro to be used in conditional expressions, such as in if/2/else expressions, or in assertions in tests.

If you are migrating from version 1.x and rely on the return value, you can enable the previous behavior by using the WaitForIt.V1.wait/2 macro instead.

Options

See the WaitForIt module documentation for further discussion of these options.

  • :timeout - the amount of time to wait (in milliseconds) before giving up
  • :pre_wait - wait for the given number of milliseconds before evaluating conditions for the first time
  • :interval - the polling interval in milliseconds, or a WaitForIt.Backoff function, at which to re-evaluate conditions (alias: :frequency)
  • :signal - disable polling and use a signal of the given name instead

Examples

Wait until the top of the hour:

WaitForIt.wait Time.utc_now.minute == 0, frequency: 60_000, timeout: 60_000 * 60

Wait up to one minute for a particular record to appear in the database:

if data = WaitForIt.wait Repo.get(Post, 42), frequency: 1000, timeout: :timer.seconds(60) do
  IO.inspect(data)
else
  IO.puts("Gave up after #{timeout} milliseconds")
end

Assert that a database record is created by some asynchronous process:

do_some_async_work()
assert %Post{id: 42} = WaitForIt.wait Repo.get(Post, 42)

wait!(expression, opts \\ [])

(macro)

The same as wait/2 but raises a WaitForIt.TimeoutError exception if it fails.

match_wait

match_wait(pattern, expression, opts \\ [])

(macro)

Wait until the given expression matches the given pattern.

Returns the value that the expression evaluated to when it matched the pattern. The pattern may include a guard, exactly like the left-hand side of a case/2 clause or a <- clause in with/1.

Where wait/2 waits for a truthy value, match_wait/3 waits for a value that matches a pattern and binds out of it. It is the most convenient form when you are waiting for a tagged result such as {:ok, value} and want value directly.

Beware bound variables in the pattern

Just like case/2, variables in the pattern are binding unless pinned with ^. The pattern is only a waiting condition, not an assertion about a specific value, so prefer guards (when ...) to express conditions on the matched value.

Options

See the WaitForIt module documentation for further discussion of these options.

  • :timeout - the amount of time to wait (in milliseconds) before giving up
  • :pre_wait - wait for the given number of milliseconds before evaluating conditions for the first time
  • :interval - the polling interval (in milliseconds) at which to re-evaluate conditions
  • :signal - disable polling and use a signal of the given name instead

Examples

Wait until a database record exists and bind it directly:

{:ok, user} = match_wait({:ok, %User{}}, Repo.fetch(User, id), timeout: 2_000)

Wait until a value satisfies a guard, then use the bound variable:

count = match_wait(n when n > 99, get_counter(), signal: :counter_wait)

On timeout (with no matching value), a MatchError is raised, exactly as if a normal pattern match had failed. Use match_wait!/3 to raise a WaitForIt.TimeoutError instead.

match_wait!(pattern, expression, opts \\ [])

(macro)

The same as match_wait/3 but raises a WaitForIt.TimeoutError exception on timeout.

case_wait

case_wait(expression, opts \\ [], blocks)

(macro)

Wait until the given expression matches one of the case clauses in the given block.

Returns the value of the matching clause, or the value of the optional else clause in the event of a timeout.

The do block passed to this macro must be a series of case clauses exactly like a built-in Elixir case/2 expression. Just like a case/2 expression, the clauses will attempt to be matched from top to bottom and the first one that matches will provide the resulting value of the expression. The difference with case_wait/3 is that if none of the clauses initially matches it will wait and periodically re-evaluate the clauses until one of them does match or a timeout occurs.

An optional else clause may also be used to provide the value in case of a timeout. If an else clause is provided and a timeout occurs, then the else clause will be evaluated and the resulting value of the else clause becomes the value of the case_wait/3 expression. If no else clause is provided and a timeout occurs, then a CaseClauseError is raised, exactly as if a normal Elixir case/2 expression were being used.

The optional else clause may also take the form of match clauses, such as those in the else clause of a with/1 expression. In this form, the else clause can match on the final value of the expression that was evaluated before the timeout occurred. See the examples below for an example of this.

Beware "catch-all" clauses

case_wait/3 expressions should not include a final "catch-all" clause, such as _, which would always match. Instead, an else clause can be used to customize the behavior and return value in the event of a waiting timeout.

See A note on "catch-all" clauses in the module docs for further information.

Options

See the WaitForIt module documentation for further discussion of these options.

  • :timeout - the amount of time to wait (in milliseconds) before giving up
  • :pre_wait - wait for the given number of milliseconds before evaluating conditions for the first time
  • :interval - the polling interval in milliseconds, or a WaitForIt.Backoff function, at which to re-evaluate conditions (alias: :frequency)
  • :signal - disable polling and use a signal of the given name instead

Examples

Wait until queue has at least 5 messages, then return them:

WaitForIt.case_wait Queue.get_messages(queue), timeout: 30_000, frequency: 100 do
  messages when length(messages) > 4 -> messages
else
  # If after 30 seconds we still don't have 5 messages, just return the messages we do have.
  messages -> messages
end

A thermostat that keeps temperature in a small range:

def thermostat(desired_temperature) do
  WaitForIt.case_wait get_current_temperature() do
    temp when temp > desired_temperature + 2 ->
      turn_on_air_conditioning()
    temp when temp < desired_temperature - 2 ->
      turn_on_heat()
  end
  thermostat(desired_temperature)
end

Wait until the process mailbox is small enough before flooding it with more messages:

WaitForIt.case_wait Process.info(stream_pid, :message_queue_len),
  frequency: 10,
  timeout: 60_000 do
  {:message_queue_len, len} when len < 500 ->
    send_chunk(conn, chunk)
else
  len ->
    raise "Timeout while sending stream response. [message_queue_len: #{len}]"
end

Production-ready

The above example is a real-world use of WaitForIt that was used to solve an issue with chunked HTTP responses using plug_cowboy. The underlying issue has since been fixed but this example is a good illustration of using WaitForIt to solve a production problem.

See https://github.com/elixir-plug/plug_cowboy/issues/10 for background and further details, if interested.

case_wait!(expression, opts \\ [], blocks)

(macro)

The same as case_wait/3 but raises a WaitForIt.TimeoutError exception if it fails.

cond_wait

cond_wait(opts \\ [], blocks)

(macro)

Wait until one of the expressions in the given block evaluates to a truthy value.

Returns the value corresponding with the matching expression, or the value of the optional else clause in the event of a timeout.

The do block passed to this macro must be a series of expressions exactly like a built-in Elixir cond/1 expression. Just like a cond/1 expression, the embedded expresions will be evaluated from top to bottom and the first one that is truthy will provide the resulting value of the expression. The difference with cond_wait/2 is that if none of the expressions is initially truthy it will wait and periodically re-evaluate them until one of them becomes truthy or a timeout occurs.

An optional else clause may also be used to provide the value in case of a timeout. If an else clause is provided and a timeout occurs, then the else clause will be evaluated and the resulting value of the else clause becomes the value of the cond_wait/2 expression. If no else clause is provided and a timeout occurs, then a CondClauseError is raised, exactly as if a normal Elixir cond/1 expression were being used.

Beware "catch-all" clauses

cond_wait/2 expressions should not include a final "catch-all" clause, such as true, which would always match. Instead, an else clause can be used to customize the behavior and return value in the event of a waiting timeout.

See A note on "catch-all" clauses in the module docs for further information.

Options

See the WaitForIt module documentation for further discussion of these options.

  • :timeout - the amount of time to wait (in milliseconds) before giving up
  • :pre_wait - wait for the given number of milliseconds before evaluating conditions for the first time
  • :interval - the polling interval in milliseconds, or a WaitForIt.Backoff function, at which to re-evaluate conditions (alias: :frequency)
  • :signal - disable polling and use a signal of the given name instead

Examples

Trigger an alarm when any sensors go beyond a threshold:

def sound_the_alarm do
  WaitForIt.cond_wait timeout: 60_000 * 60 * 24 do
    read_sensor(:sensor1) > 9 -> IO.puts("Alarm: :sensor1 too high!")
    read_sensor(:sensor2) < 100 -> IO.puts("Alarm: :sensor2 too low!")
    read_sensor(:sensor3) < 0 -> IO.puts("Alarm: :sensor3 below zero!")
  else
    IO.puts("All is good...for now.")
  end

  # Recursively call to wait for the next sensor readings...
  sound_the_alarm()
end

cond_wait!(opts \\ [], blocks)

(macro)

The same as cond_wait/2 but raises a WaitForIt.TimeoutError exception if it fails.

with_wait

with_wait(clauses, opts \\ [], blocks)

(macro)

Compose several waits in a with-style pipeline.

with_wait looks and behaves like an Elixir with/1 expression, except that its clauses can wait. The clauses are wrapped in on(...), and each clause is one of:

  • pattern <- expression - an ordinary with clause, evaluated once. If it matches, bind and continue; otherwise route to the else block.
  • pattern <~ expression - a wait-for-match clause: re-evaluate expression until it matches pattern, then bind and continue.
  • pattern <~ {expression, opts} - a wait-for-match clause with per-clause options (such as timeout:/interval:/signal:) that override the global options.

Global options for every <~ clause may be given between the on(...) wrapper and the block: with_wait on(...), timeout: 2_000 do ... end.

On success, the do block is evaluated and its value returned. If any clause fails to match — including a <~ clause that times out — control transfers to the else block (or, if there is no else, the non-matching value becomes the result), exactly like with/1. A <~ timeout is therefore indistinguishable from an ordinary non-match: the last evaluated value flows to else.

<~ precedence

<~ binds more tightly than when and the comparison operators, so guards and right-hand sides that use those operators must be parenthesized:

({:ok, n} when n > 5) <~ poll()
found <~ (Enum.find(items, &ready?/1) != nil)

Simple clauses such as {:ok, x} <~ fetch(id) and {:ok, x} <~ {fetch(id), timeout: 100} need no parentheses. For a wait dominated by a single complex condition, prefer case_wait/3.

Examples

with_wait on(
  {:ok, token}   <-  authenticate(user),
  {:ok, account} <~  {load_account(token), timeout: 2_000},
  {:ok, balance} <~  fetch_balance(account)
), interval: 50 do
  {:ok, balance}
else
  {:error, reason} -> {:error, reason}
  still_pending -> {:still_waiting, still_pending}
end

with_wait!(clauses, opts \\ [], blocks)

(macro)

The same as with_wait/3 but a <~ clause that times out raises a WaitForIt.TimeoutError instead of routing to the else block.

An else block is still honored for ordinary <- clauses that do not match.

signaling

signal(signal)

@spec signal(signal()) :: :ok

Send a signal to indicate that any processes waiting on the signal should re-evaluate their waiting conditions.

Types

signal()

@type signal() ::
  atom() | {atom(), atom()} | {atom(), atom(), atom()} | [atom()] | term()

wait_expression()

@type wait_expression() :: Macro.t()

Type to represent an expression that can be waited on.

wait_opt()

@type wait_opt() ::
  {:timeout, non_neg_integer()}
  | {:interval, non_neg_integer()}
  | {:frequency, non_neg_integer()}
  | {:pre_wait, non_neg_integer()}
  | {:signal, atom() | nil}

Options that can be used to control waiting behavior.

wait_opts()

@type wait_opts() :: [wait_opt()]

Options that can be used to control waiting behavior.

See wait_opt/0.