SearchAsh (search_ash v0.4.2)

Copy Markdown View Source

An Ash extension that adds multilingual full-text search to a resource with one search do … end block.

defmodule MyApp.Post do
  use Ash.Resource,
    domain: MyApp.Blog,
    data_layer: AshPostgres.DataLayer,
    extensions: [SearchAsh]

  search do
    fields [:title, :body]
    language_attribute :language
  end

  # ... attributes :title, :body, :language ...
end

From that block the extension generates, at compile time:

  • a :search_text string attribute (unless you defined one), holding the stemmed tokens;
  • a global change that keeps :search_text in sync on create/update, stemming each row in its own language via SearchCore;
  • a GIN expression index to_tsvector('simple', search_text) on the Postgres table — emitted into your migrations and tracked in the resource snapshot, so mix ash_postgres.generate_migrations round-trips it cleanly;
  • a :search read action taking query and language arguments, filtering on the tsvector with a tsquery built from the same pipeline (so a search for "chevaux" matches a row that stored "cheval").

Stemming happens in Elixir, so the Postgres side always uses the 'simple' configuration.

Summary

Types

What reindex_one/3 did to the index row: rebuilt it (:upserted), deleted it (:removed), flagged it archived (:archived), or found nothing to do (:noop).

Functions

Count the :global_search matches of a SearchAsh.GlobalIndex, per source_type — the numbers a results page shows on its tabs

Remove index rows whose source record no longer exists — an orphan sweep.

Backfill the unified index for all existing rows of a SearchAsh.Source resource.

Reconcile one source record's index row, by re-reading the source.

Types

reindex_result()

@type reindex_result() :: :upserted | :removed | :archived | :noop

What reindex_one/3 did to the index row: rebuilt it (:upserted), deleted it (:removed), flagged it archived (:archived), or found nothing to do (:noop).

Functions

counts_by_type(index, term, opts \\ [])

@spec counts_by_type(module(), String.t() | nil, keyword()) :: %{
  required(String.t()) => non_neg_integer()
}

Count the :global_search matches of a SearchAsh.GlobalIndex, per source_type — the numbers a results page shows on its tabs:

SearchAsh.counts_by_type(MyApp.Search.Document, "tomates", actor: user, tenant: org)
#=> %{"facture" => 12, "produit" => 3}

Runs the index's :global_search action, so everything composes as usual: the same matching (including fuzzy?), archived rows hidden, and the index's policies applied to the given :actor — a user only gets counts for what they may find.

A blank term counts everything, per type — the numbers for a results page before the user types.

Options

  • :types — the types to count (atoms or strings). nil or [] mean "not specified" (same convention as :global_search's types argument): the types actually present in the matching rows are counted, found with one extra distinct read.
  • :language, :include_archived? — forwarded to :global_search's arguments.
  • :actor, :authorize?, :tenant, :domain — as for any read.

Cost

One Ash.count per type (plus the distinct read when :types is omitted) — N small GIN-indexed counts, no hidden group-by. Fine for the handful of types a global index typically holds; pass :types to count only the tabs you display.

prune(source_resource, opts \\ [])

@spec prune(
  module(),
  keyword()
) :: non_neg_integer()

Remove index rows whose source record no longer exists — an orphan sweep.

Where reindex_one/3 reconciles one record you know changed, prune/2 reconciles a whole source in the deletion direction: it reads which of the resource's rows are still live and drops every index row that no longer has one behind it. Use it to recover from writes that deleted source rows outside Ash and were never followed by a reindex_one/3 — a bulk DELETE, a restore that went the wrong way, a botched migration:

SearchAsh.prune(MyApp.Sales.BonDeCommande, tenant: "org_42")

It streams the source (once) into the set of live source_ids, then, for each index row of this resource's source_type whose id is not in that set, applies the resource's on_destroy:remove deletes it, :archive flags it archived, exactly as reindex_one/3 would for a single gone record. Returns the number of index rows it acted on.

It only ever removes (or archives); it never adds. Pair it with reindex/2 for a full two-way reconcile — backfill missing rows, then sweep orphans.

Call it outside any transaction

Like reindex_one/3, it dispatches the index's notifications itself, so it must run outside a surrounding transaction. It reads and writes one index row per orphan, so it carries the same "built for small-to-medium tables, not a bulk-optimized job for very large datasets" caveat as reindex/2.

Why :actor and authorize?: true are rejected

For the same reason as reindex_one/3, and here the stakes are higher. prune/2 decides an index row is an orphan by finding no live source behind it. If the live set were read with a policy applied, every row that policy hides would be missing from it and pruned — so running prune/2 as a scoped user would delete the index rows of every record that user cannot see. The live set is always read with authorize?: false; absence must mean "does not exist", never "not visible to me".

Like reindex_one/3, it decides existence from the source's primary read action, so that read must return every indexable row: a plain filter on it (not base_filter) makes filtered-but-live rows look like orphans and prune would delete them. See reindex_one/3 for the full note.

A multitenant source must feed a multitenant index — prune raises otherwise. A non-multitenant index cannot be tenant-scoped, so it would hand back every tenant's rows and prune would delete the ones belonging to other tenants.

Options

  • :tenant — for a multitenant source. Scopes both the source stream and the index sweep, so prune only ever touches the tenant you name. Call once per tenant.
  • :domain — as for Ash.stream!/2.
  • :stream_with, :allow_stream_with, :batch_size, :timeout — forwarded to the source Ash.stream!/2. A read that can't keyset-stream needs stream_with: :offset (Ash streams with :keyset by default), the same option reindex/2 needs for such a resource.

It deliberately does not forward :action, :filter or anything else that would narrow which rows the stream yields — that would misclassify live rows as orphans and delete them.

reindex(source_resource, opts \\ [])

@spec reindex(
  module(),
  keyword()
) :: :ok

Backfill the unified index for all existing rows of a SearchAsh.Source resource.

Streams the source and upserts each row into its configured index. For a multitenant index, pass the tenant (call once per tenant):

SearchAsh.reindex(MyApp.Sales.BonDeCommande, tenant: "org_42")

Options are forwarded to the source read (:tenant, :domain, :authorize?, …), so you decide whose rows get backfilled; :tenant also scopes the index upsert, so mirrored rows land in the same tenant they came from.

The upsert itself is not authorized: it mirrors rows the read already let through, and the index's policies are about what a user may find, not about whether the mirror may happen.

This only ever adds rows to the index, so it cannot repair one whose source has gone away — and it reads the whole resource. To reconcile a single record after a write that bypassed Ash, use reindex_one/3; to drop index rows whose source is gone, prune/2.

Returns :ok.

reindex_one(source_resource, id, opts \\ [])

@spec reindex_one(module(), term(), keyword()) :: reindex_result()

Reconcile one source record's index row, by re-reading the source.

Use it after a write that bypassed Ash — a raw Repo.query!, a SQL cascade, a restore — which the sync and remove changes never saw:

SearchAsh.reindex_one(MyApp.Sales.BonDeCommande, id, tenant: "org_42")

It re-reads the record and reconciles, so the caller never has to work out whether the row should be added or removed:

  • the record is there → its document is rebuilt and upserted (:upserted);
  • it is gone → the resource's on_destroy decides — :remove deletes the index row (:removed), :archive keeps it flagged archived (:archived). Exactly what destroying it through Ash would have done;
  • it is gone and was never indexed → nothing to do (:noop).

Idempotent: calling it twice does what calling it once does.

Composite primary keys take a map or keyword list, as Ash.get/3 does:

SearchAsh.reindex_one(MyApp.Sales.Ligne, %{commande_id: id, numero: 2}, tenant: "org_42")

Call it after the write commits, and outside any transaction

It re-reads the source, so it must run after the bypassing write is committed and visible — otherwise it faithfully re-indexes the stale data it can still see. It also dispatches the index's notifications itself, which Ash can only do outside a transaction. Both point the same way: call it after your Repo.transaction, never inside it.

Options

  • :tenant — for a multitenant source. Scopes both the read and the index write, so the row is reconciled in the tenant it belongs to. It cannot be inferred once the record is gone, so pass it whenever reindex/2 would need it — and call once per tenant. A wrong tenant finds nothing and returns a cheerfully misleading :noop.
  • :domain — as for Ash.get/3.

Why :actor and authorize?: true are rejected

The source read always runs with authorize?: false. reindex/2 forwards :authorize? safely because it only ever upserts — an authorized read that hides rows just backfills fewer of them. Here, absence is a decision: a row a policy hid is indistinguishable from one that was deleted (both read as nil), and would be reconciled by deleting its index row. Authorization answers "may this actor see it", which must not decide whether a row exists.

authorize?: false turns off that filter and nothing else — the resource's base_filter and the tenant still apply, so an AshArchival-style soft delete is still correctly seen as gone.

The primary read action must return every indexable row

The record is read through the source's primary read action (there is deliberately no :action option), so that action decides what "exists" means here. base_filter is fine — a soft delete should read as gone. But a plain filter on the primary read is not authorization and is not turned off by authorize?: false: it always applies. A resource whose default read filters rows out (say filter expr(published == true)) will have live but filtered rows read as absent, and reindex_one/3 will remove their index rows — even though the sync change indexed them (it fires on every write, regardless of read filters). Keep the primary read unfiltered beyond base_filter, or point reindex_one/3/prune/2 at a resource whose default read returns everything.

The archived branch keeps the indexed text

For on_destroy :archive, the index row keeps its stored search_text/label and only flips archived — the source is gone, so there is nothing to rebuild from. If the text changed in the same breath as the deletion, the index retains the older one under archived: true. This is what the Ash destroy path does too.

search(body)

(macro)