Dependency-free text embedder using the hashing trick.
Tokenizes text and hashes each token into one of :dims buckets,
producing a fixed-length term-frequency vector. Cosine similarity over
these vectors approximates lexical overlap, so it needs no model and no
network — a sensible default embedder for LangEx.Store semantic search
when a neural embedding provider is not configured:
Graph.compile(builder,
store: {LangEx.Store.ETS, index: [embed: &LangEx.Embedding.Hashing.embed/1]}
)It captures word overlap, not meaning: "db is slow" and "database latency" share no tokens and score near zero. Supply a neural embedder when semantic (meaning-based) similarity matters.
Summary
Functions
Embeds text into a fixed-length term-frequency vector.