Functions for statistical calculations and analysis of numerical data.
Summary
Functions
Standard deviation as a proportion of the mean — a unitless measure of spread.
Pearson correlation coefficient between two equal-length lists, in -1..1.
Population covariance of two equal-length lists.
Running total of a list.
Counts how often each value occurs.
Geometric mean — the correct average for growth rates and ratios.
Harmonic mean — the correct average for rates over a fixed distance.
Returns the interquartile range: the spread of the middle half of the data.
Least-squares linear regression of ys on xs.
Calculates the arithmetic mean of a list of numbers.
Finds the median value from a list of numbers.
Finds the most frequently occurring value(s) in a list.
Simple moving average over a sliding window.
Rescales a list to the 0..1 range (min-max normalisation).
Values that fall outside the expected spread.
Returns the value at the given percentile using linear interpolation.
Returns the first, second and third quartiles.
Calculates the range (difference between max and min values).
The list with its outliers removed. See outliers/2 for the options.
Calculates the sample standard deviation of a dataset (divides by N-1).
Calculates the sample variance of a dataset (divides by N-1, Bessel's correction).
Calculates the population standard deviation of a dataset (divides by N).
A one-call description of a dataset.
Mean after discarding a proportion from each end.
Calculates the population variance of a dataset (divides by N).
Mean weighted by a matching list of weights.
Returns how many standard deviations value sits from the mean.
Standardises a list to zero mean and unit variance.
Functions
Standard deviation as a proportion of the mean — a unitless measure of spread.
Useful for comparing the variability of datasets with different scales or units.
Examples
iex> NumberF.Statistics.coefficient_of_variation([2, 4, 4, 4, 5, 5, 7, 9])
0.4Pearson correlation coefficient between two equal-length lists, in -1..1.
Examples
iex> NumberF.Statistics.correlation([1, 2, 3, 4, 5], [2, 4, 5, 4, 5])
0.7745966692414833Population covariance of two equal-length lists.
Examples
iex> NumberF.Statistics.covariance([1, 2, 3, 4, 5], [2, 4, 5, 4, 5])
1.2Running total of a list.
Examples
iex> NumberF.Statistics.cumulative_sum([1, 2, 3, 4])
[1, 3, 6, 10]Counts how often each value occurs.
Examples
iex> NumberF.Statistics.frequency_distribution([1, 2, 2, 3, 3, 3])
%{1 => 1, 2 => 2, 3 => 3}Geometric mean — the correct average for growth rates and ratios.
Precision
Averaging three years of 10%, 5% and 20% growth arithmetically overstates the compound result. Pass growth factors (1.10, 1.05, 1.20), not percentages.
Examples
iex> NumberF.Statistics.geometric_mean([1, 4, 16]) |> Float.round(4)
4.0Harmonic mean — the correct average for rates over a fixed distance.
Examples
iex> NumberF.Statistics.harmonic_mean([40, 60]) |> Float.round(6)
48.0Returns the interquartile range: the spread of the middle half of the data.
Examples
iex> NumberF.Statistics.iqr([1, 2, 3, 4, 5, 6, 7, 8])
3.5Least-squares linear regression of ys on xs.
Returns the slope, intercept, and the r-squared goodness of fit.
Examples
iex> NumberF.Statistics.linear_regression([1, 2, 3, 4, 5], [2, 4, 5, 4, 5])
%{slope: 0.6, intercept: 2.2, r_squared: 0.5999999999999999}Calculates the arithmetic mean of a list of numbers.
Finds the median value from a list of numbers.
Finds the most frequently occurring value(s) in a list.
Simple moving average over a sliding window.
Returns length(numbers) - window + 1 values.
Examples
iex> NumberF.Statistics.moving_average([1, 2, 3, 4, 5], 3)
[2.0, 3.0, 4.0]Rescales a list to the 0..1 range (min-max normalisation).
Not a fitted scaler
This computes its statistics from the list you pass in, so it has no fit/transform split. Scaling a test set with it does not apply the training set's scaler — it fits a new one from the test data, silently corrupting the features and leaking test-set statistics.
For a machine-learning pipeline use Scholar.Preprocessing.StandardScaler,
whose fit/2 and transform/2 are separate for exactly this reason. This
function is for exploratory work on a single dataset.
Edge cases
A list where every value is identical has no range to scale against, so it maps to all zeros rather than dividing by zero.
Examples
iex> NumberF.Statistics.normalize([10, 20, 30])
[0.0, 0.5, 1.0]Values that fall outside the expected spread.
Options: method: :iqr (default, anything beyond 1.5 x IQR from the quartiles)
or method: :zscore (anything more than three standard deviations from the mean).
Examples
iex> NumberF.Statistics.outliers([1, 2, 3, 4, 100])
[100]Returns the value at the given percentile using linear interpolation.
Precision
Uses the linear-interpolation-between-closest-ranks method (the same as NumPy's default and R's type 7). Other definitions exist and disagree on small datasets, so a percentile from here will not always match one from a spreadsheet.
Examples
iex> NumberF.Statistics.percentile([1, 2, 3, 4, 5, 6, 7, 8, 9, 10], 25)
3.25
iex> NumberF.Statistics.percentile([1, 2, 3, 4], 50)
2.5Returns the first, second and third quartiles.
Examples
iex> NumberF.Statistics.quartiles([1, 2, 3, 4, 5, 6, 7, 8])
%{q1: 2.75, q2: 4.5, q3: 6.25}Calculates the range (difference between max and min values).
The list with its outliers removed. See outliers/2 for the options.
Examples
iex> NumberF.Statistics.remove_outliers([1, 2, 3, 4, 100])
[1, 2, 3, 4]Calculates the sample standard deviation of a dataset (divides by N-1).
Use this when the list is a sample rather than the whole population. See
standard_deviation/1 for the population form.
Examples
iex> NumberF.Statistics.sample_standard_deviation([2, 4, 4, 4, 5, 5, 7, 9])
2.138089935299395Calculates the sample variance of a dataset (divides by N-1, Bessel's correction).
Examples
iex> NumberF.Statistics.sample_variance([2, 4, 4, 4, 5, 5, 7, 9])
4.571428571428571Calculates the population standard deviation of a dataset (divides by N).
Precision
This is the population form. If your list is a sample drawn from a larger
population — which is the more common case — use sample_standard_deviation/1,
which applies Bessel's correction (divides by N-1). The two differ by a factor of
sqrt(N/(N-1)), so they converge as the dataset grows but diverge sharply for
small lists.
Examples
iex> NumberF.Statistics.standard_deviation([2, 4, 4, 4, 5, 5, 7, 9])
2.0A one-call description of a dataset.
Examples
iex> NumberF.Statistics.summary([1, 2, 3, 4, 5])
%{count: 5, min: 1, q1: 2.0, median: 3, q3: 4.0, max: 5, mean: 3.0,
stddev: 1.4142135623730951}Mean after discarding a proportion from each end.
More robust than mean/1 when the extremes are unreliable.
Examples
iex> NumberF.Statistics.trimmed_mean([1, 2, 3, 4, 100], 0.2)
3.0Calculates the population variance of a dataset (divides by N).
Precision
See standard_deviation/1 — use sample_variance/1 if the list is a sample.
Examples
iex> NumberF.Statistics.variance([2, 4, 4, 4, 5, 5, 7, 9])
4.0Mean weighted by a matching list of weights.
Examples
iex> NumberF.Statistics.weighted_mean([90, 80, 70], [0.5, 0.3, 0.2])
83.0Returns how many standard deviations value sits from the mean.
Edge cases
A zero standard deviation raises: every value is identical, so "how many deviations away" has no answer rather than an answer of zero.
Examples
iex> NumberF.Statistics.z_score(6, 4, 2)
1.0Standardises a list to zero mean and unit variance.
Not a fitted scaler
This computes its statistics from the list you pass in, so it has no fit/transform split. Scaling a test set with it does not apply the training set's scaler — it fits a new one from the test data, silently corrupting the features and leaking test-set statistics.
For a machine-learning pipeline use Scholar.Preprocessing.StandardScaler,
whose fit/2 and transform/2 are separate for exactly this reason. This
function is for exploratory work on a single dataset.
Examples
iex> NumberF.Statistics.z_scores([2, 4, 6])
[-1.224744871391589, 0.0, 1.224744871391589]