Mutare.Test (mutare v0.4.1)

Copy Markdown View Source

Test helpers for projects that implement their own Mutare.Mutator.

Import this module into an ExUnit.Case to test a mutator at three levels:

The source-driven helpers all call Mutare.transform_string/2 and inherit its defaults — notably expand_uses: true, which the schema/query routing of use-heavy DSLs depends on. Each takes a trailing opts keyword list forwarded to Mutare.transform_string/2 (the mutators argument overrides any :mutators option), so a suite can thread :call_routes, :extensions, or expand_uses: false without dropping to Mutare.transform_string/2 itself.

One default differs: the source-driven helpers pass verify_invariants: true unless opts says otherwise, so every transform a test makes also checks that the mutators left the metamutant sound — each recorded mutant selectable and listed in a coverage record, none rendering unchanged, the render deterministic — and raises Mutare.InvariantError if not. A mutator that breaks one of these fails its own tests instead of silently distorting a real run's report. Pass verify_invariants: false to skip the checks, which cost a second emit pass and a parse per transform.

Selection is private to the test module

The live-mutant helpers select on a :persistent_term key private to the running ExUnit test module (isolate_selector/0 — one key per execution of the module, so two :parameterize instances are apart too), set before a metamutant is transformed or a mutant selected, so a test module using them may be async: true. A process no ExUnit test runs above must be given the key (isolate_selector/1), or the helper raises; outside ExUnit the key is the VM-wide one, and such callers must not run concurrently with one another.

import Mutare.Test in an ExUnit.Case to use them:

defmodule MyMutatorTest do
  use ExUnit.Case, async: true
  import Mutare.Test

  test "swaps + for -" do
    assert node_mutations("1 + 2", MyApp.PlusMutator) == ["1 - 2"]
  end
end

Driving a live mutant in-process

The semantic check — does the mutant actually run? — compiles a metamutant once, then flips the selection switch per id:

defmodule MyQueryTest do
  use ExUnit.Case, async: true
  import Mutare.Test

  test "the mutant changes the result, not just the source" do
    {[mod], sites} =
      compile_metamutant("defmodule Q do

def n, do: 1 + 1 end", [MyApp.PlusMutator])

    id = site_id(sites, {"1 + 1", "1 - 1"})

    assert mod.n() == 2                              # baseline
    assert with_active_mutant(id, fn -> mod.n() end) == 0   # the mutant is live
    assert mod.n() == 2                              # restored
  end
end

Summary

Types

A mutator entry the source helpers accept — anything :mutators takes: a family atom (:arithmetic), a custom module, a {module, opts} pair, or an already-resolved Mutare.Mutator.Spec.

Functions

Transforms source, compiles the complete metamutant, and returns its compiled {module, binary} pairs.

Render source to its metamutant, compile it, and return {modules, mutants}.

Returns every recorded mutation as {family_name, original_code, mutated_code}.

Returns the {original_code, mutated_code} pairs recorded for one family.

Selects, for this process, on a key private to the running ExUnit test module, and returns it.

isolate_selector/0 as a setup callback — setup_all :isolate_selector, or setup — taking the key ahead of the module's first transform and putting it in the context as :mutare_selector_key, for a process the walk cannot reach to install.

Returns the rendered metamutant source for source.

Returns the rendered node-level mutations for a parsed source snippet.

Runs fun once at baseline and once with the site matching pattern active, returning {baseline, mutated}.

Returns the single site for which predicate returns true.

Returns the id of the single site matching {original_code, mutated_code}.

Runs zero-arity fun with mutant id selected, then restores the previous selection.

Types

mutator()

@type mutator() ::
  atom() | module() | {atom() | module(), term()} | Mutare.Mutator.Spec.t()

A mutator entry the source helpers accept — anything :mutators takes: a family atom (:arithmetic), a custom module, a {module, opts} pair, or an already-resolved Mutare.Mutator.Spec.

Functions

assert_metamutant_compiles(source, mutators, opts \\ [])

@spec assert_metamutant_compiles(String.t(), [mutator()], keyword()) :: [
  {module(), binary()}
]

Transforms source, compiles the complete metamutant, and returns its compiled {module, binary} pairs.

source must contain a complete compilation unit such as a defmodule. The helper compiles it inside a unique wrapper, captures compiler output, and purges all compiled modules before returning. opts is forwarded as in diffs/3.

defmodule MyMutatorTest do
  use ExUnit.Case, async: true
  import Mutare.Test

  test "every mutant compiles" do
    assert_metamutant_compiles(
      "defmodule Sample do

def f(a, b), do: a + b end",

      [MyApp.PlusMutator]
    )
  end
end

compile_metamutant(source, mutators, opts \\ [])

@spec compile_metamutant(String.t(), [mutator()], keyword()) ::
  {[module()], [Mutare.MutationSite.t()]}

Render source to its metamutant, compile it, and return {modules, mutants}.

By default, compilation occurs inside a uniquely named wrapper module. This prevents module-name collisions and keeps ordinary self-references working. Compiled modules are purged when the test process exits.

Wrapper nesting can capture references to a real module with the same leading namespace, and it cannot resolve a forward reference to a later sibling module. Use self-contained fixtures with short module names. When top-level names are required, pass uniquify: false and manage collisions explicitly.

Each isolated compilation creates permanent module-name atoms, so this helper is intended for a bounded set of fixtures rather than an unbounded generated test.

Options are forwarded to Mutare.transform_string/2. :uniquify is consumed by this helper, and the mutators argument overrides any :mutators option.

modules are the metamutant's own compiled module atoms (the empty wrapper shell excluded), in compilation order — typically a single-element list for a single defmodule; mutants are public Mutare.MutationSite DTOs, used to resolve a mutant's id from its logical diff (site_id/2 / site_by/3). All compiled modules are purged when the test exits.

{[module], mutants} =
  compile_metamutant(
    "defmodule Q do

def n, do: 1 + 1 end",

    [MyApp.PlusMutator]
  )

assert module.n() == 2
id = site_id(mutants, {"1 + 1", "1 - 1"})
assert with_active_mutant(id, fn -> module.n() end) == 0

diffs(source, mutators, opts \\ [])

@spec diffs(String.t(), [mutator()], keyword()) :: [{atom(), String.t(), String.t()}]

Returns every recorded mutation as {family_name, original_code, mutated_code}.

This helper uses the complete transform pipeline, including name resolution, pipe handling, overlap suppression, and structural families. opts is forwarded to Mutare.transform_string/2 (the mutators argument overrides any :mutators option).

iex> import Mutare.Test
iex> diffs("def f(a, b), do: a + b", [Mutare.Mutators.Arithmetic])
[{:arithmetic, "a + b", "a - b"}]

diffs_for(source, mutators, name, opts \\ [])

@spec diffs_for(String.t(), [mutator()], atom(), keyword()) :: [
  {String.t(), String.t()}
]

Returns the {original_code, mutated_code} pairs recorded for one family.

name is the recorded family name, including any configured :as override. opts is forwarded as in diffs/3.

iex> import Mutare.Test
iex> mutators = [Mutare.Mutators.Arithmetic, Mutare.Mutators.ReturnValue]
iex> diffs_for("def f(a, b), do: a + b", mutators, :arithmetic)
[{"a + b", "a - b"}]

isolate_selector()

@spec isolate_selector() :: atom()

Selects, for this process, on a key private to the running ExUnit test module, and returns it.

The key names one execution of the module. ExUnit's runner records the running test in the process that runs the module — the parent of each test process and of the setup_all process — so the key is found by walking up from the caller, and is the same in a setup_all, in each test, and in a task a test starts: a metamutant compiled once in setup_all and selected in a test read one slot. Two executions of one module (:parameterize runs an async module once per parameter set, concurrently) have different runners and so different keys. Every helper here that transforms or selects calls it first; a test that transforms through Mutare.Transform itself calls it before doing so.

Under ExUnit, isolation is not optional: where the walk finds no test while ExUnit is running this raises, rather than fall back to the VM-wide key and share it. A process the walk cannot reach — one started under a supervisor, say — is given the key instead: isolate_selector/1 puts it in the setup context, and Process.put(Mutare.Selector.process_key(), key) in that process installs it, ahead of the walk. Outside ExUnit the process keeps the key in force (Mutare.Selector.key/0).

isolate_selector(context)

@spec isolate_selector(map()) :: %{mutare_selector_key: atom()}

isolate_selector/0 as a setup callback — setup_all :isolate_selector, or setup — taking the key ahead of the module's first transform and putting it in the context as :mutare_selector_key, for a process the walk cannot reach to install.

metamutant_source(source, mutators, opts \\ [])

@spec metamutant_source(String.t(), [mutator()], keyword()) :: String.t()

Returns the rendered metamutant source for source.

For =~ assertions on the scaffolding the transform weaves — a selector, a host's dynamic([u], …) — without destructuring Mutare.Transform.Result. opts is forwarded as in diffs/3.

metamutant = metamutant_source(source, [{Mutare.Ecto, repo: MyApp.Repo}])
assert metamutant =~ "dynamic([u]"

node_mutations(snippet, mutators)

@spec node_mutations(
  String.t(),
  module() | Mutare.Mutator.Spec.t() | [module() | Mutare.Mutator.Spec.t()]
) :: [String.t()]

Returns the rendered node-level mutations for a parsed source snippet.

This helper calls mutators directly without the transform's resolution passes or structural mutations. mutators must therefore be a module, a resolved Mutare.Mutator.Spec, or a list of either; family atoms are not accepted.

A pipe stage reaches a mutator as the direct call it is sugar for, so write the snippet that way: Enum.sort(xs) stands for xs |> Enum.sort() too.

iex> import Mutare.Test
iex> node_mutations("1 + 2", Mutare.Mutators.Arithmetic)
["1 - 2"]

iex> node_mutations("Enum.sort(xs, :desc)", Mutare.Mutators.CollectionArity)
["Enum.reverse(xs)"]

observe_mutant(sites, pattern, fun)

@spec observe_mutant([Mutare.MutationSite.t()], {pattern, pattern}, (-> result)) ::
  {result, result}
when pattern: String.t() | Regex.t(), result: var

Runs fun once at baseline and once with the site matching pattern active, returning {baseline, mutated}.

sites and pattern are as in site_id/2. The baseline runs first, and with the baseline selection pinned explicitly — so a wrong first element means the fixture is broken, not the mutant.

{baseline, mutated} =
  observe_mutant(sites, {"u.age > 18", "u.age >= 18"}, fn -> Repo.all(adults()) end)

assert boundary_user in mutated -- baseline

site_by(sites, label, pred)

Returns the single site for which predicate returns true.

label identifies the lookup in failure messages. The lookup fails and lists candidates when zero or multiple sites match.

site_id(sites, arg)

@spec site_id(
  [Mutare.MutationSite.t()],
  {pattern, pattern}
) :: pos_integer()
when pattern: String.t() | Regex.t()

Returns the id of the single site matching {original_code, mutated_code}.

A string matches exactly. A Regex matches the corresponding code field by pattern, and either side may use a different match type. The lookup fails when zero or multiple sites match and lists the candidates in the failure message.

Use site_by/3 when code matching cannot identify the site.

with_active_mutant(id, fun)

@spec with_active_mutant(non_neg_integer(), (-> result)) :: result when result: var

Runs zero-arity fun with mutant id selected, then restores the previous selection.

Selects on the test module's private key (isolate_selector/0), the one a metamutant compiled by compile_metamutant/3 in this module reads; restoration keeps sequential calls from leaking into one another.