Linear Discriminant Analysis (LDA) for classification.
LDA fits a Gaussian density to each class, assuming that all classes share the same covariance matrix. The shared covariance leads to a linear decision boundary between every pair of classes. For a sample $x$ the score of class $k$ is
$$ \delta\_{k}(x) = x^{T} \Sigma^{-1} \mu\_{k} - \frac{1}{2} \mu\_{k}^{T} \Sigma^{-1} \mu\_{k} + \log \pi\_{k} $$
where $\mu\_{k}$ is the class mean, $\pi\_{k}$ the class prior and $\Sigma$ the pooled within-class covariance. The predicted class is the one with the highest score.
The scores are computed around the overall mean $\bar{x} = \sum\_{k} \pi\_{k}
\mu\_{k}$ (as scikit-learn's svd solver does), so decision_function
differs from the expression above by a term that is constant across classes.
That term cancels in predict and predict_probability.
The pooled covariance is assumed to be invertible, which holds when the number of samples is larger than the number of features and no feature is a linear combination of the others.
Reference:
Summary
Functions
Computes the linear discriminant scores of each class for samples x.
Fits a Linear Discriminant Analysis model for sample inputs x and target
labels y.
Predicts the class of each sample in x.
Estimates class probabilities for samples x by applying a softmax to the
discriminant scores.
Functions
Computes the linear discriminant scores of each class for samples x.
Examples
iex> x = Nx.tensor([[-2.0, -1.0], [-1.0, -1.0], [-1.0, -2.0], [1.0, 1.0], [1.0, 2.0], [2.0, 1.0]])
iex> y = Nx.tensor([0, 0, 0, 1, 1, 1])
iex> model = Scholar.DiscriminantAnalysis.Linear.fit(x, y, num_classes: 2)
iex> scores = Scholar.DiscriminantAnalysis.Linear.decision_function(model, Nx.tensor([[1.5, 1.5]]))
iex> Nx.shape(scores)
{1, 2}Fits a Linear Discriminant Analysis model for sample inputs x and target
labels y.
Options
:num_classes(pos_integer/0) - Required. Number of different classes used in training. Labels must be integers in[0, num_classes).
Return Values
The function returns a struct with the following parameters:
:coefficients- Weight vector of each class, shape{num_classes, num_features}.:intercept- Intercept term of each class, shape{num_classes}.:means- Class-wise mean of the samples, shape{num_classes, num_features}.:priors- Class prior probabilities, shape{num_classes}.
Examples
iex> x = Nx.tensor([[-2.0, -1.0], [-1.0, -1.0], [-1.0, -2.0], [1.0, 1.0], [1.0, 2.0], [2.0, 1.0]])
iex> y = Nx.tensor([0, 0, 0, 1, 1, 1])
iex> model = Scholar.DiscriminantAnalysis.Linear.fit(x, y, num_classes: 2)
iex> Scholar.DiscriminantAnalysis.Linear.predict(model, Nx.tensor([[-1.5, -1.5], [1.5, 1.5]]))
#Nx.Tensor<
s32[2]
[0, 1]
>Predicts the class of each sample in x.
Examples
iex> x = Nx.tensor([[-2.0, -1.0], [-1.0, -1.0], [-1.0, -2.0], [1.0, 1.0], [1.0, 2.0], [2.0, 1.0]])
iex> y = Nx.tensor([0, 0, 0, 1, 1, 1])
iex> model = Scholar.DiscriminantAnalysis.Linear.fit(x, y, num_classes: 2)
iex> Scholar.DiscriminantAnalysis.Linear.predict(model, Nx.tensor([[-1.5, -1.5], [1.5, 1.5]]))
#Nx.Tensor<
s32[2]
[0, 1]
>Estimates class probabilities for samples x by applying a softmax to the
discriminant scores.
Examples
iex> x = Nx.tensor([[-2.0, -1.0], [-1.0, -1.0], [-1.0, -2.0], [1.0, 1.0], [1.0, 2.0], [2.0, 1.0]])
iex> y = Nx.tensor([0, 0, 0, 1, 1, 1])
iex> model = Scholar.DiscriminantAnalysis.Linear.fit(x, y, num_classes: 2)
iex> probs = Scholar.DiscriminantAnalysis.Linear.predict_probability(model, Nx.tensor([[1.5, 1.5]]))
iex> Nx.sum(probs) |> Nx.round()
#Nx.Tensor<
f32
1.0
>