Archives an experiment, optionally setting a winner variation. Archived experiments will not accept new trials.

Arguments

• experiment_id - The ID of the experiment to archive
• winner_variation - Optional winner: variation ID (integer) or name (string)

Examples

archive_experiment(123)                    # Archive without winner
archive_experiment(123, "variant_a")       # Archive with winner by name
archive_experiment(123, 456)               # Archive with winner by ID

Creates a new experiment with given name and optional description.

Returns a summary of results for a given experiment: variations, total trials, and details for both success types including counts, amounts, and rates.

Example output: [ %{

variation_id: 1,
variation_name: "Variation A",
trials: 100,
success1: %{
  count: 20,
  amount: 150.5,
  rate: 0.20,
  amount_per_trial: 1.505
},
success2: %{
  count: 15,
  amount: 75.25,
  rate: 0.15,
  amount_per_trial: 0.7525
}

}, ... ]

Formats a list of experiment results into a consistent response tuple. Returns either:

• {:ok, %{experiment_name => result}} if all successful
• {:error, %{successful: [...], failed: [...]}} if any failed

Gets all trials for a session ID across all experiments.

Gets a single experiment with its variations. Returns nil if the experiment does not exist.

Gets an experiment by name.

Gets or creates an experiment by name.

Gets or creates a trial for a session. Returns {:error, :experiment_archived} if the experiment is archived.

Gets or creates trials for multiple experiments for a session. Returns a list of {variation, status} tuples in the same order as the experiment names.

Gets or creates a trial for a user. Returns {:error, :experiment_archived} if the experiment is archived.

Gets all session trials for a user, grouped by experiment.

Gets a trial by session ID.

Gets all trials for a user, grouped by experiment.

Gets a variation by ID.

Gets a variation by experiment_name and variation name.

Links a session-based trial to a user by updating the trial's user_id. This allows tracking the same experiment across both session and user contexts.

Can be called with a list of experiment names or :all to link all session experiments.

Returns {:ok, results} on success, {:error, details} on partial/full failure.

Lists experiments with their variations preloaded.

Options

• :status - Filter by status: :active, :archived, or :all (default: :all)

Examples

list_experiments()                      # All experiments
list_experiments(status: :active)       # Only active (non-archived)
list_experiments(status: :archived)     # Only archived

Lists experiments that have trials for the given session_id.

Lists experiments that have trials for the given user_id.

Calculate a naive p-value for the difference in conversion rates among variations. For multi-variate, we might do pairwise comparisons or a more advanced approach (ANOVA, etc.). Here is a simple demonstration using a 2-proportion z-test for a pair of variations. For multiple variations, you could do them pairwise.

Records successes for multiple experiments with consistent error handling. Returns either:

• {:ok, %{experiment_name => {:ok, trial}}} if all successful
• {:error, %{successful: [...], failed: [...]}} if any failed

Records successes for session-based experiments.

Records a success for a trial.

Options

• :amount - Optional numeric value to track with the success (default: 0.0)
• :success_type - Type of success to record, either :success1 or :success2 (default: :success1)

Records a success for a trial.

Records successes for user-based experiments.

Sets a specific variation for a session trial. Returns {:ok, trial} if successful, {:error, reason} otherwise.

Unarchives an experiment, clearing the winner variation.

Updates an experiment with the given attributes.

Examples

iex> update_experiment(experiment, %{start_time: "2024-01-01", end_time: "2024-12-31"})
{:ok, %Experiment{}}

Updates the variation for a specific trial.

Allows updating variation weights for an experiment. Example:

update_variation_weights("exp_homepage", [
  {"Variation A", 1.0},
  {"Variation B", 2.0}
])

Creates or updates an experiment with variations.

Options

• :update_weights - Whether to update existing variation weights (default: true)
• :success1_label - Label for success1 metric
• :success2_label - Label for success2 metric
• :archived - Set to true to archive the experiment (only updates if explicitly provided)
• :winner - Winner variation name (string) - only used when archived: true