Benchee v0.5.0 Benchee.Statistics
Statistics related functionality that is meant to take the raw benchmark run times and then compute statistics like the average and the standard devaition.
Summary
Functions
Calculates statistical data based on a series of run times for a job in microseconds
Sorts the given jobs fastest to slowest by average
Takes a job suite with job run times, returns a map representing the statistics of the job suite as follows
Functions
Calculates statistical data based on a series of run times for a job in microseconds.
Examples
iex> run_times = [200, 400, 400, 400, 500, 500, 700, 900]
iex> Benchee.Statistics.job_statistics(run_times)
%{average: 500.0,
ips: 2000.0,
std_dev: 200.0,
std_dev_ratio: 0.4,
std_dev_ips: 800.0,
median: 450.0,
minimum: 200,
maximum: 900,
sample_size: 8}
Sorts the given jobs fastest to slowest by average.
Examples
iex> jobs = %{"Second" => %{average: 200.0}, "Third" => %{average: 400.0}, "First" => %{average: 100.0}}
iex> Benchee.Statistics.sort(jobs)
[{"First", %{average: 100.0}},
{"Second", %{average: 200.0}},
{"Third", %{average: 400.0}}]
Takes a job suite with job run times, returns a map representing the statistics of the job suite as follows:
- average - average run time of the job in μs (the lower the better)
- ips - iterations per second, how often can the given function be executed within one second (the higher the better)
- std_dev - standard deviation, a measurement how much results vary (the higher the more the results vary)
- std_dev_ratio - standard deviation expressed as how much it is relative to the average
- std_dev_ips - the absolute standard deviation of iterations per second (= ips * std_dev_ratio)
- median - when all measured times are sorted, this is the middle value (or average of the two middle values when the number of times is even). More stable than the average and somewhat more likely to be a typical you see.
- minimum - the smallest (fastest) run time measured for the job
- maximum - the biggest (slowest) run time measured for the job
- sample_size - the number of run time measurements taken
Parameters
suite
- the job suite represented as a map after running the measurements, required to have the run_times available under therun_times
key
Examples
iex> run_times = [200, 400, 400, 400, 500, 500, 700, 900]
iex> suite = %{run_times: %{"My Job" => run_times}}
iex> Benchee.Statistics.statistics(suite)
%{
statistics: %{
"My Job" => %{
average: 500.0,
ips: 2000.0,
std_dev: 200.0,
std_dev_ratio: 0.4,
std_dev_ips: 800.0,
median: 450.0,
minimum: 200,
maximum: 900,
sample_size: 8
}
},
run_times: %{"My Job" => [200, 400, 400, 400, 500, 500, 700, 900]}
}