# starfuzz A reusable, well-tested, pure Gleam toolkit for fuzzy string matching, text normalization, string distance, similarity scoring, ranking, fuzzy search, phonetic comparison, and entity resolution. Supports both **Erlang (BEAM)** and **JavaScript** compilation targets. [](https://hex.pm/packages/starfuzz) [](https://hexdocs.pm/starfuzz/) `starfuzz` provides a unified, coherent API for string similarity, search, and entity resolution in Gleam, avoiding Spotify-specific or domain-specific assumptions. ## Contents - [Start here](#start-here) - [The default configuration presets](#the-default-configuration-presets) - [Why Normalization matters](#why-normalization-matters) - [Why Tokenization matters](#why-tokenization-matters) - [Why Phonetics matter](#why-phonetics-matter) - [Distance vs. Similarity](#distance-vs-similarity) - [Why Resolver Thresholds & Confidence matter](#why-resolver-thresholds--confidence-matter) - [Quick Start Examples](#quick-start-examples) - [Module API Reference](#module-api-reference) - [Installation](#installation) - [Operational guidance](#operational-guidance) - [License](#license) ## Start here If you only need to run a basic fuzzy search over a list of strings: ```gleam import starfuzz/search pub fn search_names(query: String, candidates: List(String)) { search.strings(query, candidates) } ``` If you need a custom search configuration with minimum score and normalizer thresholds: ```gleam import starfuzz/search import starfuzz/similarity import starfuzz/normalize pub fn search_songs(query: String, songs: List(Song)) { let norm = normalize.new() |> normalize.with_lowercase |> normalize.with_trim |> normalize.with_collapsed_whitespace search.new(songs, fn(s: Song) { s.title }) |> search.using(similarity.jaro_winkler) |> search.with_normalizer(norm) |> search.with_minimum_score(0.8) |> search.with_limit(3) |> search.run(query) } ``` If you need a multi-signal ranked score composition: ```gleam import starfuzz/rank import starfuzz/similarity pub fn rank_matches(query: Song, candidates: List(Song)) { rank.builder() |> rank.add_signal("title", 0.7, fn(s: Song) { similarity.jaro(query.title, s.title) }) |> rank.add_signal("artist", 0.3, fn(s: Song) { similarity.jaro(query.artist, s.artist) }) |> rank.calculate(candidates) } ``` ## The default configuration presets The submodules are structured so you do not have to assemble distance metrics, normalizers, and tokenizers manually unless you want to. ### Defaults Table | Module | Default Configuration / Preset | |---|---| | `starfuzz/normalize` | `basic` preset (lowercase, trim, strip punctuation, collapse whitespace) | | `starfuzz/tokenize` | Whitespace word splitting | | `starfuzz/distance` | Levenshtein edit distance | | `starfuzz/similarity` | Normalized Levenshtein similarity | | `starfuzz/search` | Levenshtein similarity, no normalizer, limit None, min_score 0.0 | | `starfuzz/resolver` | threshold 0.7, confidence_thresholds #(0.75, 0.9) | --- ## Why Normalization matters Normalization transforms strings into a standard form before matching. Without normalization, simple formatting variations can completely break distance calculation. For example, `" Fear of the Dark! "` and `"fear of the dark"` are semantically identical, but Levenshtein distance treats them as 6 edits apart due to leading/trailing spaces, capital letters, and the exclamation mark. Normalizing first collapses this formatting noise. --- ## Why Tokenization matters Tokenization splits text into structural parts (characters, words, n-grams) before matching. * **Character n-grams** (e.g. bigrams, trigrams) are highly effective for catching typographical errors or spelling variants. * **Word n-grams** are useful for finding phrase overlaps while ignoring slight word reorderings. `starfuzz` offers target-independent Unicode tokenization presets out of the box. --- ## Why Phonetics matter Sometimes, words sound the same but are spelled differently (e.g., `"Robert"` and `"Rupert"`). Phonetic encoding mapping (Soundex) converts strings into structural sound representations. Comparing the Soundex codes lets you find homophone matches that edit distances might miss. --- ## Distance vs. Similarity * **Distance Metrics** (e.g., Levenshtein, Hamming) return integers where a lower value means the strings are closer. They represent the actual number of single-character operations required to turn one string into another. * **Similarity Metrics** return floats between `0.0` (entirely different) and `1.0` (identical). They are normalized so that they can be easily combined or compared across strings of different lengths. --- ## Why Resolver Thresholds & Confidence matter Entity resolution maps ambiguous queries onto a known collection of entities. Setting thresholds and confidence mappings: 1. **Reduces False Positives**: Matches below the minimum threshold are automatically filtered out. 2. **Determines Confidence**: High, Medium, and Low tiers explain how close the match was to the query. 3. **Explains Scores**: Resolvers expose the contributing weights and scores of each component (e.g., title weight vs. artist weight), making matches understandable. --- ## Quick Start Examples ### Normalizing text ```gleam import starfuzz/normalize let norm = normalize.new() |> normalize.with_lowercase |> normalize.with_trim |> normalize.without_punctuation normalize.apply(norm, " Gleam! ") // -> "gleam" ``` ### Tokenizing text ```gleam import starfuzz/tokenize tokenize.words("Fear of the Dark") // -> ["Fear", "of", "the", "Dark"] tokenize.character_ngrams("gleam", 2) // -> Ok(["gl", "le", "ea", "am"]) ``` ### Calculating similarity ```gleam import starfuzz/similarity similarity.jaro_winkler("dwayne", "duane") // -> 0.84 ``` ### Running Entity Resolution ```gleam import starfuzz/resolver import starfuzz/similarity type Song { Song(title: String, artist: String) } let song_resolver = resolver.new(fn(q: Song, c: Song) { [ resolver.component("title", 0.6, similarity.jaro_winkler(q.title, c.title)), resolver.component("artist", 0.4, similarity.jaro(q.artist, c.artist)), ] }) |> resolver.with_threshold(0.7) let query = Song("Fear of the Dark", "Iron Maiden") let candidates = [ Song("Fear of the Dark (2015 Remaster)", "Iron Maiden"), Song("Fear of the Dark", "Graveworm"), ] resolver.resolve(song_resolver, query, candidates) ``` --- ## Module API Reference | Module | Core Types | Main Functions | |---|---|---| | `starfuzz/normalize` | `Normalizer` | `new`, `with_lowercase`, `with_trim`, `apply`, `basic` | | `starfuzz/tokenize` | `TokenizeError` | `characters`, `words`, `character_ngrams`, `word_ngrams` | | `starfuzz/phonetic` | - | `soundex`, `same_soundex` | | `starfuzz/distance` | `HammingError` | `levenshtein`, `optimal_string_alignment`, `longest_common_subsequence`, `hamming` | | `starfuzz/similarity` | - | `levenshtein`, `jaro`, `jaro_winkler`, `dice`, `jaccard`, `cosine` | | `starfuzz/search` | `Match`, `SearchBuilder` | `new`, `using`, `with_normalizer`, `run`, `strings` | | `starfuzz/rank` | `ComponentScore`, `Ranking`, `RankBuilder` | `builder`, `add_signal`, `combine`, `calculate` | | `starfuzz/resolver` | `Confidence`, `Resolution`, `Resolver` | `new`, `with_threshold`, `resolve`, `component` | | `starfuzz/benchmark` | `Case`, `Algorithm`, `BenchmarkResult` | `new`, `add_algorithm`, `add_cases`, `run` | --- ## Installation Add `starfuzz` to your project: ```sh gleam add starfuzz ``` --- ## Operational guidance ### Target Independence `starfuzz` is fully compatible with both BEAM and JavaScript compiler targets. Every distance and similarity algorithm computes identical values on both targets. ### Stable Sorting All list-sorting operations are index-preserved and deterministic: when two candidate items receive the exact same score, their original index in the input list determines their relative position. ### Timing Portability `starfuzz/benchmark` uses target-specific FFIs (`os:system_time` on Erlang/BEAM and `Date.now()` on JavaScript) to compute target-independent elapsed milliseconds. --- ## License MIT License - Copyright (c) 2026 Antonio Ognio Made with ❤️ from 🇵🇪. El Perú es clave 🔑.