An Ash extension that adds multilingual full-text search to a resource with one
search do … end block.
defmodule MyApp.Post do
use Ash.Resource,
domain: MyApp.Blog,
data_layer: AshPostgres.DataLayer,
extensions: [SearchAsh]
search do
fields [:title, :body]
language_attribute :language
end
# ... attributes :title, :body, :language ...
endFrom that block the extension generates, at compile time:
- a
:search_textstring attribute (unless you defined one), holding the stemmed tokens; - a global change that keeps
:search_textin sync on create/update, stemming each row in its own language viaSearchCore; - a GIN expression index
to_tsvector('simple', search_text)on the Postgres table — emitted into your migrations and tracked in the resource snapshot, somix ash_postgres.generate_migrationsround-trips it cleanly; - a
:searchread action takingqueryandlanguagearguments, filtering on the tsvector with a tsquery built from the same pipeline (so a search for "chevaux" matches a row that stored "cheval").
Stemming happens in Elixir, so the Postgres side always uses the 'simple'
configuration.
Summary
Types
What reindex_one/3 did to the index row: rebuilt it (:upserted), deleted it
(:removed), flagged it archived (:archived), or found nothing to do (:noop).
Functions
Count the :global_search matches of a SearchAsh.GlobalIndex, per source_type —
the numbers a results page shows on its tabs
Remove index rows whose source record no longer exists — an orphan sweep.
Backfill the unified index for all existing rows of a SearchAsh.Source resource.
Reconcile one source record's index row, by re-reading the source.
Types
@type reindex_result() :: :upserted | :removed | :archived | :noop
What reindex_one/3 did to the index row: rebuilt it (:upserted), deleted it
(:removed), flagged it archived (:archived), or found nothing to do (:noop).
Functions
@spec counts_by_type(module(), String.t() | nil, keyword()) :: %{ required(String.t()) => non_neg_integer() }
Count the :global_search matches of a SearchAsh.GlobalIndex, per source_type —
the numbers a results page shows on its tabs:
SearchAsh.counts_by_type(MyApp.Search.Document, "tomates", actor: user, tenant: org)
#=> %{"facture" => 12, "produit" => 3}Runs the index's :global_search action, so everything composes as usual: the same
matching (including fuzzy?), archived rows hidden, and the index's policies
applied to the given :actor — a user only gets counts for what they may find.
A blank term counts everything, per type — the numbers for a results page before
the user types.
Options
:types— the types to count (atoms or strings).nilor[]mean "not specified" (same convention as:global_search'stypesargument): the types actually present in the matching rows are counted, found with one extradistinctread.:language,:include_archived?— forwarded to:global_search's arguments.:actor,:authorize?,:tenant,:domain— as for any read.
Cost
One Ash.count per type (plus the distinct read when :types is omitted) —
N small GIN-indexed counts, no hidden group-by. Fine for the handful of types a
global index typically holds; pass :types to count only the tabs you display.
@spec prune( module(), keyword() ) :: non_neg_integer()
Remove index rows whose source record no longer exists — an orphan sweep.
Where reindex_one/3 reconciles one record you know changed, prune/2 reconciles a whole
source in the deletion direction: it reads which of the resource's rows are still live and
drops every index row that no longer has one behind it. Use it to recover from writes that
deleted source rows outside Ash and were never followed by a reindex_one/3 — a bulk DELETE,
a restore that went the wrong way, a botched migration:
SearchAsh.prune(MyApp.Sales.BonDeCommande, tenant: "org_42")It streams the source (once) into the set of live source_ids, then, for each index row of
this resource's source_type whose id is not in that set, applies the resource's
on_destroy — :remove deletes it, :archive flags it archived, exactly as
reindex_one/3 would for a single gone record. Returns the number of index rows it acted on.
It only ever removes (or archives); it never adds. Pair it with reindex/2 for a full
two-way reconcile — backfill missing rows, then sweep orphans.
Call it outside any transaction
Like reindex_one/3, it dispatches the index's notifications itself, so it must run outside
a surrounding transaction. It reads and writes one index row per orphan, so it carries the
same "built for small-to-medium tables, not a bulk-optimized job for very large datasets"
caveat as reindex/2.
Why :actor and authorize?: true are rejected
For the same reason as reindex_one/3, and here the stakes are higher. prune/2 decides an
index row is an orphan by finding no live source behind it. If the live set were read with a
policy applied, every row that policy hides would be missing from it and pruned — so
running prune/2 as a scoped user would delete the index rows of every record that user
cannot see. The live set is always read with authorize?: false; absence must mean "does
not exist", never "not visible to me".
Like reindex_one/3, it decides existence from the source's primary read action, so
that read must return every indexable row: a plain filter on it (not base_filter) makes
filtered-but-live rows look like orphans and prune would delete them. See reindex_one/3 for
the full note.
A multitenant source must feed a multitenant index — prune raises otherwise. A non-multitenant index cannot be tenant-scoped, so it would hand back every tenant's rows and prune would delete the ones belonging to other tenants.
Options
:tenant— for a multitenant source. Scopes both the source stream and the index sweep, so prune only ever touches the tenant you name. Call once per tenant.:domain— as forAsh.stream!/2.:stream_with,:allow_stream_with,:batch_size,:timeout— forwarded to the sourceAsh.stream!/2. A read that can't keyset-stream needsstream_with: :offset(Ash streams with:keysetby default), the same optionreindex/2needs for such a resource.
It deliberately does not forward :action, :filter or anything else that would narrow
which rows the stream yields — that would misclassify live rows as orphans and delete them.
Backfill the unified index for all existing rows of a SearchAsh.Source resource.
Streams the source and upserts each row into its configured index. For a multitenant index, pass the tenant (call once per tenant):
SearchAsh.reindex(MyApp.Sales.BonDeCommande, tenant: "org_42")Options are forwarded to the source read (:tenant, :domain, :authorize?, …), so
you decide whose rows get backfilled; :tenant also scopes the index upsert, so mirrored
rows land in the same tenant they came from.
The upsert itself is not authorized: it mirrors rows the read already let through, and the index's policies are about what a user may find, not about whether the mirror may happen.
This only ever adds rows to the index, so it cannot repair one whose source has gone
away — and it reads the whole resource. To reconcile a single record after a write that
bypassed Ash, use reindex_one/3; to drop index rows whose source is gone, prune/2.
Returns :ok.
@spec reindex_one(module(), term(), keyword()) :: reindex_result()
Reconcile one source record's index row, by re-reading the source.
Use it after a write that bypassed Ash — a raw Repo.query!, a SQL cascade, a restore —
which the sync and remove changes never saw:
SearchAsh.reindex_one(MyApp.Sales.BonDeCommande, id, tenant: "org_42")It re-reads the record and reconciles, so the caller never has to work out whether the row should be added or removed:
- the record is there → its document is rebuilt and upserted (
:upserted); - it is gone → the resource's
on_destroydecides —:removedeletes the index row (:removed),:archivekeeps it flagged archived (:archived). Exactly what destroying it through Ash would have done; - it is gone and was never indexed → nothing to do (
:noop).
Idempotent: calling it twice does what calling it once does.
Composite primary keys take a map or keyword list, as Ash.get/3 does:
SearchAsh.reindex_one(MyApp.Sales.Ligne, %{commande_id: id, numero: 2}, tenant: "org_42")Call it after the write commits, and outside any transaction
It re-reads the source, so it must run after the bypassing write is committed and
visible — otherwise it faithfully re-indexes the stale data it can still see. It also
dispatches the index's notifications itself, which Ash can only do outside a transaction.
Both point the same way: call it after your Repo.transaction, never inside it.
Options
:tenant— for a multitenant source. Scopes both the read and the index write, so the row is reconciled in the tenant it belongs to. It cannot be inferred once the record is gone, so pass it wheneverreindex/2would need it — and call once per tenant. A wrong tenant finds nothing and returns a cheerfully misleading:noop.:domain— as forAsh.get/3.
Why :actor and authorize?: true are rejected
The source read always runs with authorize?: false. reindex/2 forwards :authorize?
safely because it only ever upserts — an authorized read that hides rows just backfills
fewer of them. Here, absence is a decision: a row a policy hid is indistinguishable
from one that was deleted (both read as nil), and would be reconciled by deleting its
index row. Authorization answers "may this actor see it", which must not decide whether
a row exists.
authorize?: false turns off that filter and nothing else — the resource's base_filter
and the tenant still apply, so an AshArchival-style soft delete is still correctly seen as
gone.
The primary read action must return every indexable row
The record is read through the source's primary read action (there is deliberately no
:action option), so that action decides what "exists" means here. base_filter is fine —
a soft delete should read as gone. But a plain filter on the primary read is not
authorization and is not turned off by authorize?: false: it always applies. A resource
whose default read filters rows out (say filter expr(published == true)) will have live but
filtered rows read as absent, and reindex_one/3 will remove their index rows — even though
the sync change indexed them (it fires on every write, regardless of read filters). Keep the
primary read unfiltered beyond base_filter, or point reindex_one/3/prune/2 at a resource
whose default read returns everything.
The archived branch keeps the indexed text
For on_destroy :archive, the index row keeps its stored search_text/label and only
flips archived — the source is gone, so there is nothing to rebuild from. If the text
changed in the same breath as the deletion, the index retains the older one under
archived: true. This is what the Ash destroy path does too.