Mutare.Test (mutare v0.1.0)

Copy Markdown View Source

Test helpers for projects that implement their own Mutare.Mutator.

Import this module into an ExUnit.Case to test a mutator at three levels:

The source-driven helpers all call Mutare.transform_string/2 and inherit its defaults — notably expand_uses: true, which the schema/query routing of use-heavy DSLs depends on. Each takes a trailing opts keyword list forwarded to Mutare.transform_string/2 (the mutators argument overrides any :mutators option), so a suite can thread :call_routes, :extensions, or expand_uses: false without dropping to Mutare.transform_string/2 itself.

Selection is process-global

Tests that call with_active_mutant/2 must use async: false, because the active mutant is shared across the VM.

import Mutare.Test in an ExUnit.Case to use them:

defmodule MyMutatorTest do
  use ExUnit.Case, async: true
  import Mutare.Test

  test "swaps + for -" do
    assert node_mutations("1 + 2", MyApp.PlusMutator) == ["1 - 2"]
  end
end

Driving a live mutant in-process

The semantic check — does the mutant actually run? — compiles a metamutant once, then flips the selection switch per id:

defmodule MyQueryTest do
  use ExUnit.Case, async: false
  import Mutare.Test

  test "the mutant changes the result, not just the source" do
    {[mod], sites} =
      compile_metamutant("defmodule Q do

def n, do: 1 + 1 end", [MyApp.PlusMutator])

    id = site_id(sites, {"1 + 1", "1 - 1"})

    assert mod.n() == 2                              # baseline
    assert with_active_mutant(id, fn -> mod.n() end) == 0   # the mutant is live
    assert mod.n() == 2                              # restored
  end
end

Summary

Types

A mutator entry the source helpers accept — anything :mutators takes: a family atom (:arithmetic), a custom module, a {module, opts} pair, or an already-resolved Mutare.Mutator.Spec.

Functions

Transforms source, compiles the complete metamutant, and returns its compiled {module, binary} pairs.

Render source to its metamutant, compile it, and return {modules, mutants}.

Returns every recorded mutation as {family_name, original_code, mutated_code}.

Returns the {original_code, mutated_code} pairs recorded for one family.

Returns the rendered metamutant source for source.

Returns the rendered node-level mutations for a parsed source snippet.

Runs fun once at baseline and once with the site matching pattern active, returning {baseline, mutated}.

Returns the single site for which predicate returns true.

Returns the id of the single site matching {original_code, mutated_code}.

Runs zero-arity fun with mutant id selected, then restores the previous selection.

Types

mutator()

@type mutator() ::
  atom() | module() | {atom() | module(), term()} | Mutare.Mutator.Spec.t()

A mutator entry the source helpers accept — anything :mutators takes: a family atom (:arithmetic), a custom module, a {module, opts} pair, or an already-resolved Mutare.Mutator.Spec.

Functions

assert_metamutant_compiles(source, mutators, opts \\ [])

@spec assert_metamutant_compiles(String.t(), [mutator()], keyword()) :: [
  {module(), binary()}
]

Transforms source, compiles the complete metamutant, and returns its compiled {module, binary} pairs.

source must contain a complete compilation unit such as a defmodule. The helper compiles it inside a unique wrapper, captures compiler output, and purges all compiled modules before returning. opts is forwarded as in diffs/3.

defmodule MyMutatorTest do
  use ExUnit.Case, async: true
  import Mutare.Test

  test "every mutant compiles" do
    assert_metamutant_compiles(
      "defmodule Sample do

def f(a, b), do: a + b end",

      [MyApp.PlusMutator]
    )
  end
end

compile_metamutant(source, mutators, opts \\ [])

@spec compile_metamutant(String.t(), [mutator()], keyword()) ::
  {[module()], [Mutare.MutationSite.t()]}

Render source to its metamutant, compile it, and return {modules, mutants}.

By default, compilation occurs inside a uniquely named wrapper module. This prevents module-name collisions and keeps ordinary self-references working. Compiled modules are purged when the test process exits.

Wrapper nesting can capture references to a real module with the same leading namespace, and it cannot resolve a forward reference to a later sibling module. Use self-contained fixtures with short module names. When top-level names are required, pass uniquify: false and manage collisions explicitly.

Each isolated compilation creates permanent module-name atoms, so this helper is intended for a bounded set of fixtures rather than an unbounded generated test.

Options are forwarded to Mutare.transform_string/2. :uniquify is consumed by this helper, and the mutators argument overrides any :mutators option.

modules are the metamutant's own compiled module atoms (the empty wrapper shell excluded), in compilation order — typically a single-element list for a single defmodule; mutants are public Mutare.MutationSite DTOs, used to resolve a mutant's id from its logical diff (site_id/2 / site_by/3). All compiled modules are purged when the test exits.

{[module], mutants} =
  compile_metamutant(
    "defmodule Q do

def n, do: 1 + 1 end",

    [MyApp.PlusMutator]
  )

assert module.n() == 2
id = site_id(mutants, {"1 + 1", "1 - 1"})
assert with_active_mutant(id, fn -> module.n() end) == 0

diffs(source, mutators, opts \\ [])

@spec diffs(String.t(), [mutator()], keyword()) :: [{atom(), String.t(), String.t()}]

Returns every recorded mutation as {family_name, original_code, mutated_code}.

This helper uses the complete transform pipeline, including name resolution, pipe handling, overlap suppression, and structural families. opts is forwarded to Mutare.transform_string/2 (the mutators argument overrides any :mutators option).

iex> import Mutare.Test
iex> diffs("def f(a, b), do: a + b", [Mutare.Mutators.Arithmetic])
[{:arithmetic, "a + b", "a - b"}]

diffs_for(source, mutators, name, opts \\ [])

@spec diffs_for(String.t(), [mutator()], atom(), keyword()) :: [
  {String.t(), String.t()}
]

Returns the {original_code, mutated_code} pairs recorded for one family.

name is the recorded family name, including any configured :as override. opts is forwarded as in diffs/3.

iex> import Mutare.Test
iex> mutators = [Mutare.Mutators.Arithmetic, Mutare.Mutators.ReturnValue]
iex> diffs_for("def f(a, b), do: a + b", mutators, :arithmetic)
[{"a + b", "a - b"}]

metamutant_source(source, mutators, opts \\ [])

@spec metamutant_source(String.t(), [mutator()], keyword()) :: String.t()

Returns the rendered metamutant source for source.

For =~ assertions on the scaffolding the transform weaves — a selector, a host's dynamic([u], …) — without destructuring Mutare.Transform.Result. opts is forwarded as in diffs/3.

metamutant = metamutant_source(source, [{Mutare.Ecto, repo: MyApp.Repo}])
assert metamutant =~ "dynamic([u]"

node_mutations(snippet, mutators, pipe_mode \\ :unpiped)

@spec node_mutations(
  String.t(),
  module() | Mutare.Mutator.Spec.t() | [module() | Mutare.Mutator.Spec.t()],
  :piped | :unpiped
) :: [String.t()]

Returns the rendered node-level mutations for a parsed source snippet.

This helper calls mutators directly without the transform's resolution passes or structural mutations. mutators must therefore be a module, a resolved Mutare.Mutator.Spec, or a list of either; family atoms are not accepted.

pipe_mode defaults to :unpiped. Use :piped when the snippet represents the right side of a pipe and therefore has one implicit argument.

iex> import Mutare.Test
iex> node_mutations("1 + 2", Mutare.Mutators.Arithmetic)
["1 - 2"]

iex> node_mutations("Enum.sort()", Mutare.Mutators.CollectionArity, :piped)
["Enum.reverse()"]

observe_mutant(sites, pattern, fun)

@spec observe_mutant([Mutare.MutationSite.t()], {pattern, pattern}, (-> result)) ::
  {result, result}
when pattern: String.t() | Regex.t(), result: var

Runs fun once at baseline and once with the site matching pattern active, returning {baseline, mutated}.

sites and pattern are as in site_id/2. The baseline runs first, and with the baseline selection pinned explicitly — so a wrong first element means the fixture is broken, not the mutant. Requires async: false, like with_active_mutant/2.

{baseline, mutated} =
  observe_mutant(sites, {"u.age > 18", "u.age >= 18"}, fn -> Repo.all(adults()) end)

assert boundary_user in mutated -- baseline

site_by(sites, label, pred)

Returns the single site for which predicate returns true.

label identifies the lookup in failure messages. The lookup fails and lists candidates when zero or multiple sites match.

site_id(sites, arg)

@spec site_id(
  [Mutare.MutationSite.t()],
  {pattern, pattern}
) :: pos_integer()
when pattern: String.t() | Regex.t()

Returns the id of the single site matching {original_code, mutated_code}.

A string matches exactly. A Regex matches the corresponding code field by pattern, and either side may use a different match type. The lookup fails when zero or multiple sites match and lists the candidates in the failure message.

Use site_by/3 when code matching cannot identify the site.

with_active_mutant(id, fun)

@spec with_active_mutant(non_neg_integer(), (-> result)) :: result when result: var

Runs zero-arity fun with mutant id selected, then restores the previous selection.

The selector uses VM-wide :persistent_term state. Tests that call this helper must therefore run with async: false; restoration prevents leakage between sequential calls but does not isolate concurrent processes.