View Source Mneme (Mneme v0.4.0)

/ˈniːmiː/ - Snapshot testing utilities

Hex.pm Docs CI

Mneme provides a set of familiar assertions that automate the tedious parts of testing.

This is sometimes called snapshot testing or approval testing, but that's not particulary important.

With Mneme, you write something like

auto_assert my_function()

and next time you run your tests, Mneme runs the function, generates a pattern from the returned value, prompts you to confirm it's what you expected, and updates the code for you. Now you have:

auto_assert %MyAwesomeValue{so: :cool} <- my_function()

This lets you quickly write lots of tests to ensure the behavior of your program doesn't change without you knowing. And if it does, Mneme prompts you with a diff so you can easily see what's up.

Features:

  • Automatically-maintained assertions: compare values using auto_assert, test exceptions using auto_assert_raise, or test process messages using auto_assert_receive and friends.
  • Seamless integration with ExUnit: no need to change your workflow, just run mix test.
  • Interactive prompts in your terminal when a new assertion is added or an existing one changes.
  • Syntax-aware diffs highlight the meaningful changes in a value.
  • Pattern matching auto-assertions let you assert on the parts of a value that matter and ignore the rest.

Take a brief tour:

If you'd like to see Mneme in action, you can download and run examples/tour_mneme.exs, a standalone tour that only requires that you have Elixir installed.

$ curl -o tour_mneme.exs https://raw.githubusercontent.com/zachallaun/mneme/main/examples/tour_mneme.exs

$ elixir tour_mneme.exs

setup

Setup

  1. Add :mneme do your deps in mix.exs:

    defp deps do
      [
        {:mneme, ">= 0.0.0", only: :test}
      ]
    end
  2. Add :mneme to your :import_deps in .formatter.exs:

    [
      import_deps: [:mneme],
      inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
    ]
  3. Start Mneme right after you start ExUnit in test/test_helper.exs:

    ExUnit.start()
    Mneme.start()
  4. Add use Mneme wherever you use ExUnit.Case:

    defmodule MyTest do
      use ExUnit.Case, async: true
      use Mneme
    
      test "arithmetic" do
        # use auto_assert instead of ExUnit's assert - run this test
        # and delight in all the typing you don't have to do
        auto_assert 2 + 2
      end
    end

generated-patterns

Generated patterns

Mneme tries to generate match patterns that are close to what you would write. Basic data types like numbers, lists, tuples, and strings will generate as you would expect. For maps and structs, Mneme will often give you a couple of options. For values without a literal representation, like pids, guards will be used.

auto_assert pid when is_pid(pid) <- self()

Additionally, local bindings can be found and pinned as a part of patterns. This keeps the number of "magic values" down and makes tests more robust.

test "create_post/1 creates a new post with valid attrs", %{user: user} do
  valid_attrs = %{title: "my_post", author: user}

  auto_assert create_post(valid_attrs)
end

# generates:

test "create_post/1 creates a new post with valid attrs", %{user: user} do
  valid_attrs = %{title: "my_post", author: user}

  auto_assert {:ok, %Post{title: "my_post", author: ^user}} <- create_post(valid_attrs)
end

non-exhaustive-list-of-special-cases

Non-exhaustive list of special cases

  • Pinned variables are generated by default if a value is equal to a variable in scope.

  • Date and time values are written using their sigil representation.

  • Struct patterns only include fields that are different from the struct defaults.

  • Structs defined by Ecto schemas exclude primary keys, association foreign keys, and auto generated fields like :inserted_at and :updated_at. This is because these fields are often randomly generated and would fail on subsequent tests.

formatting-required

Formatting required

Mneme uses Rewrite to update source code, formatting that code before saving the file. Currently, the Elixir formatter and FreedomFormatter are supported. If you do not use a formatter, the first auto-assertion will reformat the entire file.

continuous-integration

Continuous Integration

In a CI environment, Mneme will not attempt to prompt and update any assertions, but will instead fail any tests that would update. This behavior is enabled by the CI environment variable, which is set by convention by many continuous integration providers.

export CI=true

editor-support

Editor support

Guides for optional editor integration can be found here:

configuration

Configuration

Mneme supports a variety of flexible configuration options that can be applied at multiple levels of granularity, from your entire test suite down to an individual test. While the default behavior will work well for the majority of cases, it's worth knowing which levers and knobs you have available to tweak Mneme to fit your individual workflow.

options

Options

  • :action (:prompt | :accept | :reject) - The action to be taken when an auto-assertion updates. If CI=true is set in environment variables, the action will always be :reject. The default value is :prompt.

  • :default_pattern (:infer | :first | :last) - The default pattern to be selected if prompted to update an assertion. The default value is :infer.

  • :diff (:text | :semantic) - Controls the diff engine used to display changes when an auto- assertion updates. If :semantic, uses a custom diff engine to highlight only meaningful changes in the value. If :text, uses the Myers Difference algorithm to highlight all changes in text. The default value is :semantic.

  • :diff_style (:side_by_side | :stacked) - Controls how diffs are rendered when the :diff option is set to :semantic. If :side_by_side, old and new code will be rendered side-by-side if the terminal has sufficient space. If :stacked, old and new code will be rendered one on top of the other. The default value is :side_by_side.

  • :force_update (boolean/0) - Setting to true will force auto-assertions to update even when they would otherwise succeed. This can be especially helpful when adding new keys to maps or structs since a pattern like %{} would not normally prompt as the match still succeeds. The default value is false.

  • :target (:mneme | :ex_unit) - The target output for auto-assertions. If :mneme, the expression will remain an auto-assertion. If :ex_unit, the expression will be rewritten as an ExUnit assertion. The default value is :mneme.

configuring-mneme

Configuring Mneme

There are four ways you can apply configuration options; Each is more specific than the last and will override any conflicting options that were set prior.

  • When calling Mneme.start/1, which will apply to the entire test run;
  • When calling use Mneme, which will apply to all tests in that module;
  • In a @mneme_describe module attribute, which will apply to all tests that follow in the given ExUnit.Case.describe/2 block;
  • In a @mneme module attribute, which will apply only to the next test that follows.

For instance, when an auto-assertion has multiple possible patterns available, Mneme will try to infer the best one to show you first. If you always want the last (and usually most complex) generated pattern, you could call Mneme.start/1 like this:

# test/test_helper.exs
ExUnit.start()
Mneme.start(default_pattern: :last)

As mentioned above, this can be overriden at the module-level, in a describe block, or for an individual test:

defmodule MyTest do
  use ExUnit.Case
  use Mneme, default_pattern: :infer

  test "the default pattern will exclude :baz when this runs" do
    map = %{foo: :one, baz: :three}
    auto_assert %{foo: 1, bar: 2} <- Map.put(map, bar: :two)
  end

  describe "..." do
    @mneme_describe action: :reject

    test "fails without prompting" do
      auto_assert :wrong <- MyModule.some_fun()
    end

    @mneme action: :prompt
    test "prompts to update" do
      auto_assert :wrong <- MyModule.another_fun()
    end
  end
end

Breaking up with Mneme? While the official stance of the library is that this is Not Recommended™, Mneme can even convert all of your existing auto-assertions to regular assertions:

Mneme.start(
  target: :ex_unit,
  force_update: true,
  action: :accept
)

Probably don't do this, but if you do, make sure all your tests are committed first in case you want to get back together.


use Mneme

When you use Mneme in a test module, assertions are imported and module attributes are made available for configuration.

Link to this section Summary

Setup

Starts Mneme to run auto-assertions as they appear in your tests.

Link to this section Setup

Starts Mneme to run auto-assertions as they appear in your tests.

This will almost always be added to your test/test_helper.exs, just below the call to ExUnit.start():

# test/test_helper.exs
ExUnit.start()
Mneme.start()

options

Options

  • :restart (boolean) - Restarts Mneme if it has previously been started. This option enables certain IEx-based testing workflows that allow tests to be run without a startup penalty. Defaults to false.

  • Any option defined in the Configuration section of the module docs.

Link to this section Assertions

Link to this macro

auto_assert(expression)

View Source (macro)

Pattern-generating variant of ExUnit.Assertions.assert/1.

examples

Examples

auto_assert generates assertions when tests run, issuing a terminal prompt before making any changes (unless configured otherwise).

auto_assert [1, 2] ++ [3, 4]

# after running the test and accepting the change
auto_assert [1, 2, 3, 4] <- [1, 2] ++ [3, 4]

If the match no longer succeeds, a warning and new prompt will be issued to update it to the new value.

auto_assert [1, 2, 3, 4] <- [1, 2] ++ [:a, :b]

# after running the test and accepting the change
auto_assert [1, 2, :a, :b] <- [1, 2] ++ [:a, :b]

Prompts are only issued if the pattern doesn't match the value, so that pattern can also be changed manually.

# this assertion succeeds, so no prompt is issued
auto_assert [1, 2, | _] <- [1, 2] ++ [:a, :b]

differences-from-exunit-assert

Differences from ExUnit assert

The auto_assert macro is meant to match assert very closely, but there are a few differences to note:

  • Pattern-matching assertions use the <- operator instead of the = match operator.

  • Unlike ExUnit's assert, auto_assert can match falsy values. The following are equivalent:

    falsy = nil
    auto_assert nil <- falsy
    assert falsy == nil
  • Guards can be added with a when clause, while assert would require a second assertion. For example:

    auto_assert pid when is_pid(pid) <- self()
    
    assert pid = self()
    assert is_pid(pid)
  • Bindings in an auto_assert are not available outside of that assertion. For example:

    auto_assert pid when is_pid(pid) <- self()
    pid # ERROR: pid is not bound

    If you need to use the result of the assertion, it will evaluate to the expression's value.

    pid = auto_assert pid when is_pid(pid) <- self()
    pid # pid is the result of self()
Link to this macro

auto_assert_raise(function)

View Source (since 0.3.0) (macro)

See auto_assert_raise/3.

Link to this macro

auto_assert_raise(exception, function)

View Source (since 0.3.0) (macro)

See auto_assert_raise/3.

Link to this macro

auto_assert_raise(exception, message, function)

View Source (since 0.3.0) (macro)

Pattern-generating variant of ExUnit.Assertions.assert_raise/3.

If the given function does not raise, the assertion will fail.

Like auto_assert/1, you will be prompted to automatically update the assertion if the raised raised exception changes.

examples

Examples

You can pass an anonymous function that takes no arguments and is expected to raise an exception.

auto_assert_raise fn ->
  some_call_expected_to_raise()
end

# after running the test and accepting changes
auto_assert_raise Some.Exception, fn ->
  some_call_expected_to_raise()
end

# optionally include the message
auto_assert_raise Some.Exception, "perhaps with a message", fn ->
  some_call_expected_to_raise()
end

A captured function of arity zero can also be used.

auto_assert_raise &some_call_expected_to_raise/0

# after running the test and accepting changes
auto_assert_raise Some.Exception, &some_call_expected_to_raise/0
Link to this macro

auto_assert_receive()

View Source (since 0.3.0) (macro)

See auto_assert_receive/2.

Link to this macro

auto_assert_receive(pattern)

View Source (since 0.3.0) (macro)

See auto_assert_receive/2.

Link to this macro

auto_assert_receive(pattern, timeout)

View Source (since 0.3.0) (macro)

Pattern-generating variant of ExUnit.Assertions.assert_receive/3.

timeout is in milliseconds and defaults to 100.

examples

Examples

Process.send_after(self(), {:some, :message}, 50)

auto_assert_receive()

# after running the test, messages appearing within 100ms
# will be available as options
auto_assert_receive {:some, :message}

A custom timeout can be specified as a second argument.

Process.send_after(self(), {:some, :message}, 150)

auto_assert_receive nil, 300

# messages appearing within 300ms will now appear as options
auto_assert_receive {:some, :message}, 300
Link to this macro

auto_assert_received()

View Source (since 0.3.0) (macro)

See auto_assert_received/1.

Link to this macro

auto_assert_received(pattern)

View Source (since 0.3.0) (macro)

Pattern-generating variant of ExUnit.Assertions.assert_received/2.

Similar to auto_assert_receive/2, except that the timeout is set to 0, so the expected message must already be in the current process' mailbox.

examples

Examples

send(self(), {:some, :message})

auto_assert_received()

# after running the test, messages in the current process
# inbox will be available as options
auto_assert_receive {:some, :message}