SearchAsh (search_ash v0.6.0)

Copy Markdown View Source

An Ash extension that adds multilingual full-text search to a resource with one search do … end block.

defmodule MyApp.Post do
  use Ash.Resource,
    domain: MyApp.Blog,
    data_layer: AshPostgres.DataLayer,
    extensions: [SearchAsh]

  search do
    fields [:title, :body]
    language_attribute :language
  end

  # ... attributes :title, :body, :language ...
end

From that block the extension generates, at compile time:

  • a :search_text string attribute (unless you defined one), holding the stemmed tokens;
  • a global change that keeps :search_text in sync on create/update, stemming each row in its own language via SearchCore;
  • a GIN expression index to_tsvector('simple', search_text) on the Postgres table — emitted into your migrations and tracked in the resource snapshot, so mix ash_postgres.generate_migrations round-trips it cleanly;
  • a :search read action taking query and language arguments, filtering on the tsvector with a tsquery built from the same pipeline (so a search for "chevaux" matches a row that stored "cheval").

Stemming happens in Elixir, so the Postgres side always uses the 'simple' configuration.

Summary

Types

What reindex_one/3 did to the index row: rebuilt it (:upserted), deleted it (:removed), flagged it archived (:archived), or found nothing to do (:noop).

Functions

Count the :global_search matches of a SearchAsh.GlobalIndex per value of one index column — the numbers that populate a results page's filter sidebar (facets)

Count the :global_search matches of a SearchAsh.GlobalIndex, per source_type — the numbers a results page shows on its tabs

Remove index rows whose source record no longer exists — an orphan sweep.

Backfill the unified index for all existing rows of a SearchAsh.Source resource.

Reconcile one source record's index row, by re-reading the source.

Types

reindex_result()

@type reindex_result() :: :upserted | :removed | :archived | :noop

What reindex_one/3 did to the index row: rebuilt it (:upserted), deleted it (:removed), flagged it archived (:archived), or found nothing to do (:noop).

Functions

counts_by(index, term, attribute, opts \\ [])

@spec counts_by(module(), String.t() | nil, atom(), keyword()) :: %{
  required(term()) => non_neg_integer()
}

Count the :global_search matches of a SearchAsh.GlobalIndex per value of one index column — the numbers that populate a results page's filter sidebar (facets):

SearchAsh.counts_by(MyApp.Search.Document, "tomates", :statut, tenant: org)
#=> %{"envoyee" => 5, "payee" => 2}

# A date column bucketed by month (:day, :month or :year):
SearchAsh.counts_by(MyApp.Search.Document, "", :document_date, bucket: :month, tenant: org)
#=> %{~D[2026-06-01] => 3, ~D[2026-07-01] => 8}

attribute is any column on the index, typically one filled by a source's index_attribute (a status, a category, a date, …). It runs the index's :global_search action, so everything composes as usual: the same matching (including fuzzy?), archived rows hidden, and the index's policies applied to the given :actor — a user only gets counts for what they may find. A blank term counts everything, per value.

Options

  • :bucket — for a date column, group by :day, :month or :year instead of by exact date; each key is the start of its bucket (~D[2026-07-01] for July). Omit it to count by exact value.
  • :values — the values to count. Omitted (the default) counts the values present in the matching rows. Pass it to fix the set, and to bound the cost on a wider column (see below). With :bucket, these are bucket starts (~D[2026-07-01]).
  • :language, :include_archived? — forwarded to :global_search's arguments.
  • :actor, :authorize?, :tenant, :domain — as for any read.

What is counted

A scalar column is counted per exact value (status, type, category — the fit). An array column (a tag list) is counted per element: a row is counted under every tag it holds, via has/2. A date column with :bucket is counted per period. A row whose value is nil (or an empty list) is not counted under any value. Amounts and un-bucketed dates count per exact value (numeric ranges are not yet available).

Cost

One Ash.count per value — the full-text match is re-run each time (a single GROUP BY would be faster but would bypass the index's policies, so it is deliberately not used; see the roadmap). Cheap for the handful of values a facet dimension holds; on a high-cardinality column, pass :values to count only the ones you display. Discovering the present values of an array or bucketed date column reads (and holds in memory) the matching rows — to flatten their lists / truncate their dates in Elixir — so on a broad or blank query over a large index, :values is the guard against a memory/latency cliff.

counts_by_type(index, term, opts \\ [])

@spec counts_by_type(module(), String.t() | nil, keyword()) :: %{
  required(String.t()) => non_neg_integer()
}

Count the :global_search matches of a SearchAsh.GlobalIndex, per source_type — the numbers a results page shows on its tabs:

SearchAsh.counts_by_type(MyApp.Search.Document, "tomates", actor: user, tenant: org)
#=> %{"facture" => 12, "produit" => 3}

A thin wrapper over counts_by/4 on the :source_type column — see it for how the counting composes with matching, policies and archived rows.

Options

  • :types — the types to count (atoms or strings). nil or [] mean "not specified" (same convention as :global_search's types argument): the types actually present in the matching rows are counted.
  • :language, :include_archived? — forwarded to :global_search's arguments.
  • :actor, :authorize?, :tenant, :domain — as for any read.

prune(source_resource, opts \\ [])

@spec prune(
  module(),
  keyword()
) :: non_neg_integer()

Remove index rows whose source record no longer exists — an orphan sweep.

Where reindex_one/3 reconciles one record you know changed, prune/2 reconciles a whole source in the deletion direction: it reads which of the resource's rows are still live and drops every index row that no longer has one behind it. Use it to recover from writes that deleted source rows outside Ash and were never followed by a reindex_one/3 — a bulk DELETE, a restore that went the wrong way, a botched migration:

SearchAsh.prune(MyApp.Sales.BonDeCommande, tenant: "org_42")

It streams the source (once) into the set of live source_ids, then, for each index row of this resource's source_type whose id is not in that set, applies the resource's on_destroy:remove deletes it, :archive flags it archived, exactly as reindex_one/3 would for a single gone record. Returns the number of index rows it acted on.

It only ever removes (or archives); it never adds. Pair it with reindex/2 for a full two-way reconcile — backfill missing rows, then sweep orphans.

Call it outside any transaction

Like reindex_one/3, it dispatches the index's notifications itself, so it must run outside a surrounding transaction. It reads and writes one index row per orphan, so it carries the same "built for small-to-medium tables, not a bulk-optimized job for very large datasets" caveat as reindex/2.

Why :actor and authorize?: true are rejected

For the same reason as reindex_one/3, and here the stakes are higher. prune/2 decides an index row is an orphan by finding no live source behind it. If the live set were read with a policy applied, every row that policy hides would be missing from it and pruned — so running prune/2 as a scoped user would delete the index rows of every record that user cannot see. The live set is always read with authorize?: false; absence must mean "does not exist", never "not visible to me".

Like reindex_one/3, it decides existence from the source's primary read action, so that read must return every indexable row: a plain filter on it (not base_filter) makes filtered-but-live rows look like orphans and prune would delete them. See reindex_one/3 for the full note.

A multitenant source must feed a multitenant index — prune raises otherwise. A non-multitenant index cannot be tenant-scoped, so it would hand back every tenant's rows and prune would delete the ones belonging to other tenants.

Options

  • :tenant — for a multitenant source. Scopes both the source stream and the index sweep, so prune only ever touches the tenant you name. Call once per tenant.
  • :domain — as for Ash.stream!/2.
  • :stream_with, :allow_stream_with, :batch_size, :timeout — forwarded to the source Ash.stream!/2. A read that can't keyset-stream needs stream_with: :offset (Ash streams with :keyset by default), the same option reindex/2 needs for such a resource.

It deliberately does not forward :action, :filter or anything else that would narrow which rows the stream yields — that would misclassify live rows as orphans and delete them.

reindex(source_resource, opts \\ [])

@spec reindex(
  module(),
  keyword()
) :: :ok

Backfill the unified index for all existing rows of a SearchAsh.Source resource.

Streams the source and upserts each row into its configured index. For a multitenant index, pass the tenant (call once per tenant):

SearchAsh.reindex(MyApp.Sales.BonDeCommande, tenant: "org_42")

Options are forwarded to the source read (:tenant, :domain, :authorize?, …), so you decide whose rows get backfilled; :tenant also scopes the index upsert, so mirrored rows land in the same tenant they came from.

The upsert itself is not authorized: it mirrors rows the read already let through, and the index's policies are about what a user may find, not about whether the mirror may happen.

This only ever adds rows to the index, so it cannot repair one whose source has gone away — and it reads the whole resource. To reconcile a single record after a write that bypassed Ash, use reindex_one/3; to drop index rows whose source is gone, prune/2.

Returns :ok.

reindex_one(source_resource, id, opts \\ [])

@spec reindex_one(module(), term(), keyword()) :: reindex_result()

Reconcile one source record's index row, by re-reading the source.

Use it after a write that bypassed Ash — a raw Repo.query!, a SQL cascade, a restore — which the sync and remove changes never saw:

SearchAsh.reindex_one(MyApp.Sales.BonDeCommande, id, tenant: "org_42")

It re-reads the record and reconciles, so the caller never has to work out whether the row should be added or removed:

  • the record is there → its document is rebuilt and upserted (:upserted);
  • it is gone → the resource's on_destroy decides — :remove deletes the index row (:removed), :archive keeps it flagged archived (:archived). Exactly what destroying it through Ash would have done;
  • it is gone and was never indexed → nothing to do (:noop).

Idempotent: calling it twice does what calling it once does.

Composite primary keys take a map or keyword list, as Ash.get/3 does:

SearchAsh.reindex_one(MyApp.Sales.Ligne, %{commande_id: id, numero: 2}, tenant: "org_42")

Call it after the write commits, and outside any transaction

It re-reads the source, so it must run after the bypassing write is committed and visible — otherwise it faithfully re-indexes the stale data it can still see. It also dispatches the index's notifications itself, which Ash can only do outside a transaction. Both point the same way: call it after your Repo.transaction, never inside it.

Options

  • :tenant — for a multitenant source. Scopes both the read and the index write, so the row is reconciled in the tenant it belongs to. It cannot be inferred once the record is gone, so pass it whenever reindex/2 would need it — and call once per tenant. A wrong tenant finds nothing and returns a cheerfully misleading :noop.
  • :domain — as for Ash.get/3.

Why :actor and authorize?: true are rejected

The source read always runs with authorize?: false. reindex/2 forwards :authorize? safely because it only ever upserts — an authorized read that hides rows just backfills fewer of them. Here, absence is a decision: a row a policy hid is indistinguishable from one that was deleted (both read as nil), and would be reconciled by deleting its index row. Authorization answers "may this actor see it", which must not decide whether a row exists.

authorize?: false turns off that filter and nothing else — the resource's base_filter and the tenant still apply, so an AshArchival-style soft delete is still correctly seen as gone.

The primary read action must return every indexable row

The record is read through the source's primary read action (there is deliberately no :action option), so that action decides what "exists" means here. base_filter is fine — a soft delete should read as gone. But a plain filter on the primary read is not authorization and is not turned off by authorize?: false: it always applies. A resource whose default read filters rows out (say filter expr(published == true)) will have live but filtered rows read as absent, and reindex_one/3 will remove their index rows — even though the sync change indexed them (it fires on every write, regardless of read filters). Keep the primary read unfiltered beyond base_filter, or point reindex_one/3/prune/2 at a resource whose default read returns everything.

The archived branch keeps the indexed text

For on_destroy :archive, the index row keeps its stored search_text/label and only flips archived — the source is gone, so there is nothing to rebuild from. If the text changed in the same breath as the deletion, the index retains the older one under archived: true. This is what the Ash destroy path does too.

search(body)

(macro)