LiveLoad.Scenario behaviour (LiveLoad v0.0.1-rc.51)

Copy Markdown View Source

A behaviour module for implementing a load testing scenario to be run via LiveLoad.

A LiveLoad.Scenario runs in a distributed fashion. Nodes are elastically created via LiveLoad whenever a load test is being performed. On each node, the config/1 callback is called once, to initialize the node's configuration, and a set of user processes are created and run to simulate the load by calling the scenario's run/3 callback with the current user ID and the config that was created for this node.

use LiveLoad.Scenario

In order for LiveLoad to be able to run your scenario correctly, you must use LiveLoad.Scenario. This will not only set the behaviour for LiveLoad, but also set up various functionality related to running the scenario properly through LiveLoad's internals. LiveLoad's internals are opaque, and while you can see how they work by browsing the code and reading the docs, they should not be assumed to be stable. Ensuring that you are creating scenarios with use LiveLoad.Scenario will make certain that any scenario you create will be stable and runnable by LiveLoad.

The config/1 callback

When you use LiveLoad.Scenario, the LiveLoad.Scenario module will define an empty config/1 function for you which will return an empty map. If you don't need any configurable parts in your scenario, this injected callback can remain and does not need to be overriden by your scenario module.

Scenario Lifecycle

When a LiveLoad.Scenario is run, the config/1 callback is called once per node, before any user processes are created. The value returned from config/1 is then passed into the run/3 callback on every iteration of the scenario for every user.

After config/1 returns, LiveLoad creates the configured number of user processes for this node. Each user process is given its own LiveLoad.Browser.Context and LiveLoad.Scenario.Context, and the user process enters a loop that calls the scenario's run/3 callback over and over until the configured :scenario_duration has been reached.

Each iteration of the loop runs run/3 with the current LiveLoad.Scenario.Context and the config that was returned from config/1. The LiveLoad.Scenario.Context returned from run/3 becomes the context for the next iteration, allowing values to be carried forward via LiveLoad.Scenario.Context.assign/3. See LiveLoad.Scenario.Context for more details on how the context is maintained across iterations.

If run/3 returns a context that is halted? or failed?, or if it raises an exception, the user process will be terminated and no further iterations will run for that user. Any other users on the node will continue running their own iterations until the :scenario_duration is reached.

Once the :scenario_duration has elapsed, the user processes will finish their current iteration and then terminate. The scenario is considered complete once all user processes on all nodes have terminated.

Summary

Types

config()

A config that is created by the config/1 callback on initialization of the LiveLoad.Scenario.

t()

Any module implementing the LiveLoad.Scenario behaviour.

user_id()

The user ID passed to the run/3 callback. It can be either an integer or a binary.

Callbacks

config(opts)

Invoked once per node that LiveLoad is running the load test on. config/1 will receive any options passed in to LiveLoad.run/1 that have not been consumed yet and can return a config value that will be passed in to run/3.

run(context, user_id, config)

Invoked once per user. run/3 is the actual load test that will be run, measured, and instrumented by LiveLoad.

throttles(config)

Define any throttles that the load test will need.

Types

config()

@type config() :: map() | keyword() | struct() | term()

A config that is created by the config/1 callback on initialization of the LiveLoad.Scenario.

A config can be any term, as it will simply be passed into the run/3 callback and can be handled by the scenario.

t()

@type t() :: module()

Any module implementing the LiveLoad.Scenario behaviour.

user_id()

@type user_id() :: integer() | binary()

The user ID passed to the run/3 callback. It can be either an integer or a binary.

Callbacks

config(opts)

@callback config(opts :: keyword()) :: {:ok, config()} | {:error, term()}

Invoked once per node that LiveLoad is running the load test on. config/1 will receive any options passed in to LiveLoad.run/1 that have not been consumed yet and can return a config value that will be passed in to run/3.

config/1 is called synchronously on startup, and if an error is returned, the entire load test will fail for this scenario.

run(context, user_id, config)

@callback run(
  context :: LiveLoad.Scenario.Context.t(),
  user_id :: user_id(),
  config :: config()
) ::
  LiveLoad.Scenario.Context.t() | {:error, term()}

Invoked once per user. run/3 is the actual load test that will be run, measured, and instrumented by LiveLoad.

throttles(config)

@callback throttles(config()) :: [
  LiveLoad.Scenario.Throttle.Rate.t()
  | LiveLoad.Scenario.Throttle.Interarrival.t()
  | LiveLoad.Scenario.Throttle.Parallelism.t()
]

Define any throttles that the load test will need.

Throttles are cluster wide throttling and rate limiting mechanisms that enforce a smoothed, non-bursty execution rate through the throttle. If you configure a LiveLoad.Scenario.Throttle.Rate throttle to 100 per minute, you get one execution approximately once every 600ms, not 100 executions in the first moment of each minute.

There are 3 types of built-in throttles that each enable different mechanisms:

Throttles must be named with a specific name and can be utilized in a run/3 callback through a LiveLoad.Scenario.Context.throttle/2 call with the same name.

Documentation about how to configure each rate limiter can be found in their respective moduledocs.