View Source Search (search v0.3.0)

⚡ Fast full-text search for Elixir

This library provides simple, fast, in-memory full-text search functionality for Elixir applications.

Features

  • 🧠 Memory efficient indexing of documents
  • 🔎 Exact match search
  • 🏃 Prefix search
  • 🧩 Fuzzy search
  • 🔢 Modern search result ranking algorithm
  • 🔀 Add and remove documents anytime

Usage

Creating an Index

To create a new index, use the new/1 function with a list of fields to be indexed:

index = Search.new(fields: [:title, :content])

Adding Documents

To add a document to the index, use the add/2 function with the index and the document:

document = %{id: 1, title: "Elixir", content: "Elixir is a dynamic, functional language."}
index = Search.add!(index, document)

You can also add multiple documents at once:

documents = [
  %{id: 2, title: "Phoenix", content: "Phoenix is a web framework for Elixir."},
  %{id: 3, title: "Nerves", content: "Nerves is a framework for embedded systems."}
]
index = Search.add!(index, documents)

Removing Documents

To remove a document from the index, use the remove/2 function with the index and the document:

index = Search.remove!(index, document)

You can also remove multiple documents at once:

index = Search.remove!(index, documents)

Searching

To search the index, use the search/3 function with the index and the query string:

Search.search(index, "web famewrk", prefix?: true, fuzzy?: true)
[
  %{
    id: 2,
    matches: %{"framework" => [:content], "web" => [:content]},
    fields: %{},
    score: 1.6965399945163802,
    terms: ["web", "framework"]
  },
  %{
    id: 3,
    matches: %{"framework" => [:content]},
    fields: %{},
    score: 0.24367025800793077,
    terms: ["framework"]
  }
]

Internals

The library uses a Radix tree for efficient indexing and retrieval of terms. It also implements the BM25 algorithm for relevance scoring and the Levenstein distance algorithm for calculating edit distances.

Summary

Functions

Adds a document or a list of documents to the index.

Adds a document or a list of documents to the index like add/2, but raises if an error occurs.

Creates a new search index.

Removes a document or a list of documents from the index. The document(s) must be unchanged.

Removes a document or a list of documents from the index like remove/2, but raises if an error occurs.

Searches the index for documents matching the given query string.

Functions

@spec add(Search.Index.t(), map() | [map()]) ::
  {:ok, Search.Index.t()} | {:error, String.t()}

Adds a document or a list of documents to the index.

Parameters

  • index: The current index state.
  • document: A map representing a single document or a list of such maps.

Returns

  • The updated index with the new document(s) added.

Examples

iex> index = Search.new(fields: [:title, :content])
iex> document = %{id: 1, title: "Elixir", content: "Elixir is a dynamic, functional language."}
iex> {:ok, index} = Search.add(index, document)
iex> documents = [
...>   %{id: 2, title: "Phoenix", content: "Phoenix is a web framework for Elixir."},
...>   %{id: 3, title: "Nerves", content: "Nerves is a framework for embedded systems."}
...> ]
iex> {:ok, index} = Search.add(index, documents)
@spec add!(Search.Index.t(), map() | [map()]) :: Search.Index.t()

Adds a document or a list of documents to the index like add/2, but raises if an error occurs.

@spec new(Keyword.t()) :: Search.Index.t()

Creates a new search index.

Parameters

  • opts: A keyword list of options.
    • fields - A list of fields to be indexed.
    • return_field_data - (Optional) A list of fields to be returned with search results.

Returns

  • A new %Index{} struct.

Examples

iex> index = Search.new(fields: [:title, :content])
Link to this function

remove(error, documents)

View Source
@spec remove(Search.Index.t(), map() | [map()]) ::
  {:ok, Search.Index.t()} | {:error, String.t()}

Removes a document or a list of documents from the index. The document(s) must be unchanged.

Parameters

  • index: The current index state.
  • document: A map representing a single document or a list of such maps to be removed.

Returns

  • The updated index with the document(s) removed.

Examples

iex> index = Search.new(fields: [:title, :content])
iex> document = %{id: 1, title: "Elixir", content: "Elixir is a dynamic, functional language."}
iex> index = Search.add!(index, document)
iex> {:ok, index} = Search.remove(index, document)
iex> documents = [
...>   %{id: 2, title: "Phoenix", content: "Phoenix is a web framework for Elixir."},
...>   %{id: 3, title: "Nerves", content: "Nerves is a framework for embedded systems."}
...> ]
iex> index = Search.add!(index, documents)
iex> {:ok, index} = Search.remove(index, documents)
Link to this function

remove!(index, document)

View Source
@spec remove!(Search.Index.t(), map() | [map()]) :: Search.Index.t()

Removes a document or a list of documents from the index like remove/2, but raises if an error occurs.

Link to this function

search(index, query_string, opts \\ [])

View Source
@spec search(Search.Index.t(), String.t(), Keyword.t()) :: [map()]

Searches the index for documents matching the given query string.

Parameters

  • index: The current index state.
  • query_string: The search query string.
  • opts: The search options.
    • prefix?: Whether to perform a prefix search. Defaults to false.
    • fuzzy?: Whether to perform a fuzzy search. Defaults to false.
    • fuzziness: The fuzziness to use for the fuzzy search; i.e. the maximum edit distance between terms being compared. Defaults to 2.
    • weights: The weights to use for the search ranking. Defaults to [prefix: 0.375, fuzzy: 0.45].

Returns

  • A list of maps representing the search results, sorted by relevance score in descending order. Each map contains:
    • id - The document ID.
    • fields - The fields returned with the search results.
    • score - The relevance score of the document.
    • terms - The terms that matched in the document.
    • matches - The fields in which the terms matched.

Examples

iex> index = Search.new(fields: [:title, :content])
iex> document = %{id: 1, title: "Elixir", content: "Elixir is a dynamic, functional language."}
iex> index = Search.add!(index, document)
iex> Search.search(index, "Elixir")
[
  %{
    id: 1,
    fields: %{},
    score: 0.8630462173553426,
    terms: ["elixir"],
    matches: %{"elixir" => [:title, :content]}
  }
]
iex> Search.search(index, "Eli", prefix?: true)
[
  %{
    id: 1,
    fields: %{},
    score: 0.28142811435500303,
    terms: ["elixir"],
    matches: %{"elixir" => [:title, :content]}
  }
]
iex> Search.search(index, "lixir", fuzzy?: true)
[
  %{
    id: 1,
    fields: %{},
    score: 0.27740771272136017,
    terms: ["elixir"],
    matches: %{"elixir" => [:title, :content]}
  }
]