Latu.ML.Feature.MinHashLSHModel (latu_ml v0.2.0)

Copy Markdown View Source

Attributes of a fitted MinHashLSHModel.

Every accessor here is one name on the server's allowlist for org.apache.spark.ml.feature.MinHashLSHModel — 2 of them — so tab completion is that allowlist. The server refuses anything else with CONNECT_ML.ATTRIBUTE_NOT_ALLOWED; these refuse a model of the wrong class before it gets that far, and name the module that would have taken it.

Allowed by the server but not here: toString is Identifiable's, and tells you less than the model's own uid. Latu.ML.attribute/2 and Latu.ML.attribute/3 will send any of these names; it is what comes back that has nowhere to go.

The allowlist is inherited rather than per class: this one is the union of org.apache.spark.ml.feature.LSHModel, org.apache.spark.ml.util.Identifiable.

Summary

Functions

approx_nearest_neighbors(holder, dataset, key, num_nearest_neighbors, dist_col)

Given a large dataset and an item, approximately find at most k items which have the closest distance to the item. If the outputCol is missing, the method will transform the data; if the outputCol exists, it will use that. This allows caching of the transformed data when necessary.

approx_similarity_join(holder, dataset_a, dataset_b, threshold, dist_col)

Join two datasets to approximately find all pairs of rows whose distance are smaller than the threshold. If the outputCol is missing, the method will transform the data; if the outputCol exists, it will use that. This allows caching of the transformed data when necessary.

class()

The JVM class these accessors belong to.

Functions

approx_nearest_neighbors(holder, dataset, key, num_nearest_neighbors, dist_col)

@spec approx_nearest_neighbors(
  Latu.ML.Model.t(),
  Latu.DataFrame.t(),
  Nx.Tensor.t() | Latu.ML.SparseVector.t(),
  integer(),
  String.t() | atom()
) :: Latu.DataFrame.t()

Given a large dataset and an item, approximately find at most k items which have the closest distance to the item. If the outputCol is missing, the method will transform the data; if the outputCol exists, it will use that. This allows caching of the transformed data when necessary.

A lazy builder: the Fetch rides a relation, so this hands back a Latu.DataFrame and nothing has run until you collect it.

Arguments

approx_similarity_join(holder, dataset_a, dataset_b, threshold, dist_col)

@spec approx_similarity_join(
  Latu.ML.Model.t(),
  Latu.DataFrame.t(),
  Latu.DataFrame.t(),
  number(),
  String.t() | atom()
) :: Latu.DataFrame.t()

Join two datasets to approximately find all pairs of rows whose distance are smaller than the threshold. If the outputCol is missing, the method will transform the data; if the outputCol exists, it will use that. This allows caching of the transformed data when necessary.

A lazy builder: the Fetch rides a relation, so this hands back a Latu.DataFrame and nothing has run until you collect it.

Arguments

  • dataset_a — a Latu.DataFrame, sent as a relation rather than a literal
  • dataset_b — a Latu.DataFrame, sent as a relation rather than a literal
  • threshold — a number, sent as a double
  • dist_col — a string, or an atom

class()

@spec class() :: String.t()

The JVM class these accessors belong to.