Scientist v0.2.0 Scientist.Experiment behaviour
A behaviour module for creating and running experiments.
An experiment contains all information about how your control and candidate functions operate, and how their observations are reported. Experiments include functionality for determining when they should run and how they behave when exceptions are thrown.
The macros exposed by Scientist
are a thin wrapper around the functions in this
module. If you would like to, you can use the corresponding functions to create
and run your experiment.
In addition to the required callbacks, you can also define custom defaults and exception handling behaviour.
Custom Defaults
default_name/0
and default_context/0
determine the default name and context,
respectively, of unconfigured experiments in your module.
Custom Exception Handling
raised/3
and thrown/3
determine how your experiment will handle exceptions
during an operation specified by the user. They receive the experiment as well
as the operation name and exception. When left unspecified, exceptions thrown
during an operation will be unhandled by Scientist
.
The following operations report exceptions:
:enabled
:compare
:clean
:ignore
:run_if
Summary
Functions
Adds fun
to the experiment as a candidate
Adds fun
to the experiment as the control
Adds a function to the experiment that is used to clean observed values
Adds a function to the experiment that is used to compare observations
Adds an ignore function to the experiment
Creates an experiment
Returns true if the given observations match
Runs the experiment
Returns the value of the experiment’s run_if function
Adds a function to the experiment that should only execute when the experiment is run
Adds a function to the experiment that is used to determine if it should run
Returns true if a mismatch should be ignored
Returns true if the experiment should run
Macros
Executes the given block, reporting exceptions
Functions
Adds fun
to the experiment as a candidate.
Raises Scientist.DuplicateError
if the experiment already has a candidate with name
.
Adds fun
to the experiment as the control.
Raises Scientist.DuplicateError
if the experiment already has a control.
Adds a function to the experiment that is used to clean observed values.
When handling observations, the result of cleaner
will be available under cleaned_value
.
If an exception is thrown in cleaner
, it will be reported through the thrown
and raised
callbacks as operation :clean
.
Adds a function to the experiment that is used to compare observations.
If an exception is thrown in compare_fn
, it will be reported through the thrown
and raised
callbacks as operation :compare
.
Adds an ignore function to the experiment.
The experiment will ignore a mismatch whenever this function returns true. There is no limit on the number of ignore functions that can be configured.
If an exception is thrown in ignore_fn
, it will be reported through the thrown
and raised
callbacks as operation :ignore
.
Creates an experiment.
Creates an experiment with the name
and opts
.
The following options are available:
:module
- The callback module to use, defaults toScientist.Default
.:context
- A map of values to be stored in an observation, defaults to%{}
.:raise_on_mismatches
- Iftrue
, any mismatches in this experiment’s observations will raise aScientist.MismatchError
, defaults tofalse
.
Returns true if the given observations match.
This uses the experiment’s compare function, if any. If none is configured,
==/2
will be used.
Reports a :compare
error to the callback module if an exception is caught.
Runs the experiment.
If enabled?/0
or a configured run_if function return a falsey value, the experiment
will not be run and only the control will be executed.
Raises Scientist.MissingControlError
if the experiment has no control.
Raises Scientist.MismatchError
if the experiment has mismatched observations and is
configured with raise_on_mismatched: true
.
Returns the value of the experiment’s run_if function.
If the experiment has no run_if function configured, true
is returned.
Reports a :run_if
error to the callback module if an exception is caught.
Adds a function to the experiment that should only execute when the experiment is run.
Adds a function to the experiment that is used to determine if it should run.
If this function returns false
, the experiment will not be run.
If an exception is thrown in run_if_fn
, it will be reported through the thrown
and raised
callbacks as operation :run_if
.
Returns true if a mismatch should be ignored.
Runs each of the configured ignore functions in turn, ignoring a mismatch when
any of them return true
.
Reports an :ignore
error to the callback module if an exception is caught.
Macros
Callbacks
Specs
enabled? :: Boolean | nil
Returns true
if the experiment should be run.
If a falsey value is returned, the candidate blocks of the experiment will be ignored, only running the control.