View Source Search (search v0.2.0)
⚡ Fast full-text search for Elixir
This library provides simple, fast, in-memory full-text search functionality for Elixir applications.
👷 Warning
This library is built on a solid foundation but is still under heavy development.
Features
- 🧠 Memory efficient indexing of documents
- 🔎 Exact match search
- 🏃 Prefix search
- 🔜 Fuzzy search
- 🔜 Auto-suggestion engine
- 🔢 Modern search result ranking algorithm
- 🔀 Add and remove documents anytime
Usage
Creating an Index
To create a new index, use the new/1
function with a list of fields to be
indexed:
index = Search.new(fields: [:title, :content])
Adding Documents
To add a document to the index, use the add/2
function with the index and
the document:
document = %{id: 1, title: "Elixir", content: "Elixir is a dynamic, functional language."}
index = Search.add!(index, document)
You can also add multiple documents at once:
documents = [
%{id: 2, title: "Phoenix", content: "Phoenix is a web framework for Elixir."},
%{id: 3, title: "Nerves", content: "Nerves is a framework for embedded systems."}
]
index = Search.add!(index, documents)
Removing Documents
To remove a document from the index, use the remove/2
function with the
index and the document:
index = Search.remove!(index, document)
You can also remove multiple documents at once:
index = Search.remove!(index, documents)
Searching
To search the index, use the search/3
function with the index and the query
string:
Search.search(index, "Elixir")
[
%{
id: 1,
matches: %{"elixir" => [:title, :content]},
fields: %{},
terms: ["elixir"],
score: 2.194907312448878
},
%{
id: 2,
matches: %{"elixir" => [:content]},
fields: %{},
terms: ["elixir"],
score: 0.6962007371655166
}
]
Internals
The library uses a Radix tree for efficient indexing and retrieval of terms. It also implements the BM25 algorithm for relevance scoring.
Summary
Functions
Adds a document or a list of documents to the index.
Adds a document or a list of documents to the index like add/2
, but raises
if an error occurs.
Creates a new search index.
Removes a document or a list of documents from the index. The document(s) must be unchanged.
Removes a document or a list of documents from the index like remove/2
, but raises
if an error occurs.
Searches the index for documents matching the given query string.
Functions
@spec add(Search.Index.t(), map() | [map()]) :: {:ok, Search.Index.t()} | {:error, String.t()}
Adds a document or a list of documents to the index.
Parameters
index
: The current index state.document
: A map representing a single document or a list of such maps.
Returns
- The updated index with the new document(s) added.
Examples
iex> index = Search.new(fields: [:title, :content])
iex> document = %{id: 1, title: "Elixir", content: "Elixir is a dynamic, functional language."}
iex> {:ok, index} = Search.add(index, document)
iex> documents = [
...> %{id: 2, title: "Phoenix", content: "Phoenix is a web framework for Elixir."},
...> %{id: 3, title: "Nerves", content: "Nerves is a framework for embedded systems."}
...> ]
iex> {:ok, index} = Search.add(index, documents)
@spec add!(Search.Index.t(), map() | [map()]) :: Search.Index.t()
Adds a document or a list of documents to the index like add/2
, but raises
if an error occurs.
@spec new(Keyword.t()) :: Search.Index.t()
Creates a new search index.
Parameters
- opts: A keyword list of options.
fields
- A list of fields to be indexed.return_field_data
- (Optional) A list of fields to be returned with search results.
Returns
- A new
%Index{}
struct.
Examples
iex> index = Search.new(fields: [:title, :content])
@spec remove(Search.Index.t(), map() | [map()]) :: {:ok, Search.Index.t()} | {:error, String.t()}
Removes a document or a list of documents from the index. The document(s) must be unchanged.
Parameters
index
: The current index state.document
: A map representing a single document or a list of such maps to be removed.
Returns
- The updated index with the document(s) removed.
Examples
iex> index = Search.new(fields: [:title, :content])
iex> document = %{id: 1, title: "Elixir", content: "Elixir is a dynamic, functional language."}
iex> index = Search.add!(index, document)
iex> {:ok, index} = Search.remove(index, document)
iex> documents = [
...> %{id: 2, title: "Phoenix", content: "Phoenix is a web framework for Elixir."},
...> %{id: 3, title: "Nerves", content: "Nerves is a framework for embedded systems."}
...> ]
iex> index = Search.add!(index, documents)
iex> {:ok, index} = Search.remove(index, documents)
@spec remove!(Search.Index.t(), map() | [map()]) :: Search.Index.t()
Removes a document or a list of documents from the index like remove/2
, but raises
if an error occurs.
@spec search(Search.Index.t(), String.t(), Keyword.t()) :: [map()]
Searches the index for documents matching the given query string.
Parameters
index
: The current index state.query_string
: The search query string.opts
: The search options.prefix?
: Whether to perform a prefix search. Defaults tofalse
.weights
: The weights to use for the search. Defaults to[prefix: 0.375]
.
Returns
- A list of maps representing the search results, sorted by relevance score in descending order. Each map contains:
id
- The document ID.fields
- The fields returned with the search results.score
- The relevance score of the document.terms
- The terms that matched in the document.matches
- The fields in which the terms matched.
Examples
iex> index = Search.new(fields: [:title, :content])
iex> document = %{id: 1, title: "Elixir", content: "Elixir is a dynamic, functional language."}
iex> index = Search.add!(index, document)
iex> Search.search(index, "Elixir")
[
%{
id: 1,
fields: %{},
score: 0.8630462173553426,
terms: ["elixir"],
matches: %{"elixir" => [:title, :content]}
}
]
iex> Search.search(index, "Eli", prefix?: true)
[
%{
id: 1,
fields: %{},
score: 0.28142811435500303,
terms: ["elixir"],
matches: %{"elixir" => [:title, :content]}
}
]