Scholar.Decomposition.KernelPCA (Scholar v0.5.0)

Kernel Principal Component Analysis (Kernel PCA).

Kernel PCA is an extension of Scholar.Decomposition.PCA that performs the decomposition in a reproducing kernel Hilbert space instead of the original feature space. Replacing the inner product with a kernel function lets the method capture non-linear structure that ordinary PCA cannot.

The time complexity is $O(N^3)$ where $N$ is the number of samples, since it relies on the eigendecomposition of the $N \times N$ kernel matrix.

References:

[1] Schölkopf, B., Smola, A., & Müller, K. R. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural computation, 10(5), 1299-1319.

Summary

Functions

fit(x, opts \\ [])

Fits a Kernel PCA for sample inputs x.

fit_transform(x, opts \\ [])

Fits a Kernel PCA on x and projects x onto the principal components.

transform(model, x)

For a fitted model projects samples x onto the principal components.

Functions

fit(x, opts \\ [])

Fits a Kernel PCA for sample inputs x.

Options

:num_components (pos_integer/0) - Required. The number of principal components to keep.
:kernel - The kernel used to compute the pairwise similarities between samples. The default value is :linear.
:gamma - Kernel coefficient for the :rbf, :poly and :sigmoid kernels. When nil it defaults to 1 / num_features. The default value is nil.
:degree (pos_integer/0) - Degree of the :poly kernel. The default value is 3.
:coef0 - Independent term of the :poly and :sigmoid kernels. The default value is 1.0.

Return Values

The function returns a struct with the following parameters:

:eigenvalues - The eigenvalues of the centered kernel matrix, sorted in decreasing order.
:eigenvectors - The eigenvectors of the centered kernel matrix.
:x_fit - The training data, kept to compute the kernel against new samples.
:kernel_fit_rows - Column means of the training kernel matrix, used to center the kernel of new samples.
:kernel_fit_all - Mean of the whole training kernel matrix.
:kernel, :gamma, :degree, :coef0 - The kernel options used to fit the model, applied again when computing the kernel of new samples.

Examples

iex> x = Nx.tensor([[0.5, 0.2, 0.8], [1.0, 0.5, 0.2], [0.3, 1.0, 0.7], [0.9, 0.1, 1.0]])
iex> kpca = Scholar.Decomposition.KernelPCA.fit(x, num_components: 2, kernel: :rbf)
iex> kpca.eigenvalues
Nx.tensor([0.3644834, 0.26074564])

fit_transform(x, opts \\ [])

Fits a Kernel PCA on x and projects x onto the principal components.

Options

:num_components (pos_integer/0) - Required. The number of principal components to keep.
:kernel - The kernel used to compute the pairwise similarities between samples. The default value is :linear.
:gamma - Kernel coefficient for the :rbf, :poly and :sigmoid kernels. When nil it defaults to 1 / num_features. The default value is nil.
:degree (pos_integer/0) - Degree of the :poly kernel. The default value is 3.
:coef0 - Independent term of the :poly and :sigmoid kernels. The default value is 1.0.

Return Values

The function returns a tensor with the decomposed data.

Examples

iex> x = Nx.tensor([[0.5, 0.2, 0.8], [1.0, 0.5, 0.2], [0.3, 1.0, 0.7], [0.9, 0.1, 1.0]])
iex> Scholar.Decomposition.KernelPCA.fit_transform(x, num_components: 2, kernel: :rbf)
Nx.tensor(
  [
    [-0.13543777, -0.16535977],
    [0.021126157, 0.44115686],
    [0.4689568, -0.15726776],
    [-0.3546452, -0.1185294]
  ]
)

transform(model, x)

For a fitted model projects samples x onto the principal components.

Return Values

The function returns a tensor with the decomposed data.

Examples

iex> x = Nx.tensor([[0.5, 0.2, 0.8], [1.0, 0.5, 0.2], [0.3, 1.0, 0.7], [0.9, 0.1, 1.0]])
iex> kpca = Scholar.Decomposition.KernelPCA.fit(x, num_components: 2, kernel: :rbf)
iex> Scholar.Decomposition.KernelPCA.transform(kpca, Nx.tensor([[0.5, 0.5, 0.5]]))
Nx.tensor(
  [
    [0.12497942, 0.029221743]
  ]
)