View Source Mneme (Mneme v0.4.0)
/ˈniːmiː/ - Snapshot testing utilities
Mneme provides a set of familiar assertions that automate the tedious parts of testing.
This is sometimes called snapshot testing or approval testing, but that's not particulary important.
With Mneme, you write something like
auto_assert my_function()
and next time you run your tests, Mneme runs the function, generates a pattern from the returned value, prompts you to confirm it's what you expected, and updates the code for you. Now you have:
auto_assert %MyAwesomeValue{so: :cool} <- my_function()
This lets you quickly write lots of tests to ensure the behavior of your program doesn't change without you knowing. And if it does, Mneme prompts you with a diff so you can easily see what's up.
Features:
- Automatically-maintained assertions: compare values using
auto_assert
, test exceptions usingauto_assert_raise
, or test process messages usingauto_assert_receive
and friends. - Seamless integration with ExUnit: no need to change your workflow, just run
mix test
. - Interactive prompts in your terminal when a new assertion is added or an existing one changes.
- Syntax-aware diffs highlight the meaningful changes in a value.
- Pattern matching auto-assertions let you assert on the parts of a value that matter and ignore the rest.
Take a brief tour:
If you'd like to see Mneme in action, you can download and run examples/tour_mneme.exs, a standalone tour that only requires that you have Elixir installed.
$ curl -o tour_mneme.exs https://raw.githubusercontent.com/zachallaun/mneme/main/examples/tour_mneme.exs
$ elixir tour_mneme.exs
setup
Setup
Add
:mneme
do your deps inmix.exs
:defp deps do [ {:mneme, ">= 0.0.0", only: :test} ] end
Add
:mneme
to your:import_deps
in.formatter.exs
:[ import_deps: [:mneme], inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"] ]
Start Mneme right after you start ExUnit in
test/test_helper.exs
:ExUnit.start() Mneme.start()
Add
use Mneme
wherever youuse ExUnit.Case
:defmodule MyTest do use ExUnit.Case, async: true use Mneme test "arithmetic" do # use auto_assert instead of ExUnit's assert - run this test # and delight in all the typing you don't have to do auto_assert 2 + 2 end end
generated-patterns
Generated patterns
Mneme tries to generate match patterns that are close to what you would write. Basic data types like numbers, lists, tuples, and strings will generate as you would expect. For maps and structs, Mneme will often give you a couple of options. For values without a literal representation, like pids, guards will be used.
auto_assert pid when is_pid(pid) <- self()
Additionally, local bindings can be found and pinned as a part of patterns. This keeps the number of "magic values" down and makes tests more robust.
test "create_post/1 creates a new post with valid attrs", %{user: user} do
valid_attrs = %{title: "my_post", author: user}
auto_assert create_post(valid_attrs)
end
# generates:
test "create_post/1 creates a new post with valid attrs", %{user: user} do
valid_attrs = %{title: "my_post", author: user}
auto_assert {:ok, %Post{title: "my_post", author: ^user}} <- create_post(valid_attrs)
end
non-exhaustive-list-of-special-cases
Non-exhaustive list of special cases
Pinned variables are generated by default if a value is equal to a variable in scope.
Date and time values are written using their sigil representation.
Struct patterns only include fields that are different from the struct defaults.
Structs defined by Ecto schemas exclude primary keys, association foreign keys, and auto generated fields like
:inserted_at
and:updated_at
. This is because these fields are often randomly generated and would fail on subsequent tests.
formatting-required
Formatting required
Mneme uses Rewrite
to update source code, formatting that code before saving the file.
Currently, the Elixir formatter and FreedomFormatter
are supported.
If you do not use a formatter, the first auto-assertion will reformat the entire file.
continuous-integration
Continuous Integration
In a CI environment, Mneme will not attempt to prompt and update any assertions, but will instead fail any tests that would update.
This behavior is enabled by the CI
environment variable, which is set by convention by many continuous integration providers.
export CI=true
editor-support
Editor support
Guides for optional editor integration can be found here:
configuration
Configuration
Mneme supports a variety of flexible configuration options that can be applied at multiple levels of granularity, from your entire test suite down to an individual test. While the default behavior will work well for the majority of cases, it's worth knowing which levers and knobs you have available to tweak Mneme to fit your individual workflow.
options
Options
:action
(:prompt | :accept | :reject
) - The action to be taken when an auto-assertion updates. IfCI=true
is set in environment variables, the action will always be:reject
. The default value is:prompt
.:default_pattern
(:infer | :first | :last
) - The default pattern to be selected if prompted to update an assertion. The default value is:infer
.:diff
(:text | :semantic
) - Controls the diff engine used to display changes when an auto- assertion updates. If:semantic
, uses a custom diff engine to highlight only meaningful changes in the value. If:text
, uses the Myers Difference algorithm to highlight all changes in text. The default value is:semantic
.:diff_style
(:side_by_side | :stacked
) - Controls how diffs are rendered when the:diff
option is set to:semantic
. If:side_by_side
, old and new code will be rendered side-by-side if the terminal has sufficient space. If:stacked
, old and new code will be rendered one on top of the other. The default value is:side_by_side
.:force_update
(boolean/0
) - Setting totrue
will force auto-assertions to update even when they would otherwise succeed. This can be especially helpful when adding new keys to maps or structs since a pattern like%{}
would not normally prompt as the match still succeeds. The default value isfalse
.:target
(:mneme | :ex_unit
) - The target output for auto-assertions. If:mneme
, the expression will remain an auto-assertion. If:ex_unit
, the expression will be rewritten as an ExUnit assertion. The default value is:mneme
.
configuring-mneme
Configuring Mneme
There are four ways you can apply configuration options; Each is more specific than the last and will override any conflicting options that were set prior.
- When calling
Mneme.start/1
, which will apply to the entire test run; - When calling
use Mneme
, which will apply to all tests in that module; - In a
@mneme_describe
module attribute, which will apply to all tests that follow in the givenExUnit.Case.describe/2
block; - In a
@mneme
module attribute, which will apply only to the next test that follows.
For instance, when an auto-assertion has multiple possible patterns
available, Mneme will try to infer the best one to show you first.
If you always want the last (and usually most complex) generated
pattern, you could call Mneme.start/1
like this:
# test/test_helper.exs
ExUnit.start()
Mneme.start(default_pattern: :last)
As mentioned above, this can be overriden at the module-level, in a
describe
block, or for an individual test:
defmodule MyTest do
use ExUnit.Case
use Mneme, default_pattern: :infer
test "the default pattern will exclude :baz when this runs" do
map = %{foo: :one, baz: :three}
auto_assert %{foo: 1, bar: 2} <- Map.put(map, bar: :two)
end
describe "..." do
@mneme_describe action: :reject
test "fails without prompting" do
auto_assert :wrong <- MyModule.some_fun()
end
@mneme action: :prompt
test "prompts to update" do
auto_assert :wrong <- MyModule.another_fun()
end
end
end
Breaking up with Mneme? While the official stance of the library is that this is Not Recommended™, Mneme can even convert all of your existing auto-assertions to regular assertions:
Mneme.start(
target: :ex_unit,
force_update: true,
action: :accept
)
Probably don't do this, but if you do, make sure all your tests are committed first in case you want to get back together.
use Mneme
When you
use Mneme
in a test module, assertions are imported and module attributes are made available for configuration.
Link to this section Summary
Setup
Starts Mneme to run auto-assertions as they appear in your tests.
Assertions
Pattern-generating variant of ExUnit.Assertions.assert/1
.
Pattern-generating variant of ExUnit.Assertions.assert_raise/3
.
Pattern-generating variant of ExUnit.Assertions.assert_receive/3
.
Pattern-generating variant of ExUnit.Assertions.assert_received/2
.
Link to this section Setup
Starts Mneme to run auto-assertions as they appear in your tests.
This will almost always be added to your test/test_helper.exs
, just
below the call to ExUnit.start()
:
# test/test_helper.exs
ExUnit.start()
Mneme.start()
options
Options
:restart
(boolean) - Restarts Mneme if it has previously been started. This option enables certain IEx-based testing workflows that allow tests to be run without a startup penalty. Defaults tofalse
.Any option defined in the Configuration section of the module docs.
Link to this section Assertions
Pattern-generating variant of ExUnit.Assertions.assert/1
.
examples
Examples
auto_assert
generates assertions when tests run, issuing a terminal
prompt before making any changes (unless configured otherwise).
auto_assert [1, 2] ++ [3, 4]
# after running the test and accepting the change
auto_assert [1, 2, 3, 4] <- [1, 2] ++ [3, 4]
If the match no longer succeeds, a warning and new prompt will be issued to update it to the new value.
auto_assert [1, 2, 3, 4] <- [1, 2] ++ [:a, :b]
# after running the test and accepting the change
auto_assert [1, 2, :a, :b] <- [1, 2] ++ [:a, :b]
Prompts are only issued if the pattern doesn't match the value, so that pattern can also be changed manually.
# this assertion succeeds, so no prompt is issued
auto_assert [1, 2, | _] <- [1, 2] ++ [:a, :b]
differences-from-exunit-assert
Differences from ExUnit assert
The auto_assert
macro is meant to match assert
very closely, but
there are a few differences to note:
Pattern-matching assertions use the
<-
operator instead of the=
match operator.Unlike ExUnit's
assert
,auto_assert
can match falsy values. The following are equivalent:falsy = nil auto_assert nil <- falsy assert falsy == nil
Guards can be added with a
when
clause, whileassert
would require a second assertion. For example:auto_assert pid when is_pid(pid) <- self() assert pid = self() assert is_pid(pid)
Bindings in an
auto_assert
are not available outside of that assertion. For example:auto_assert pid when is_pid(pid) <- self() pid # ERROR: pid is not bound
If you need to use the result of the assertion, it will evaluate to the expression's value.
pid = auto_assert pid when is_pid(pid) <- self() pid # pid is the result of self()
See auto_assert_raise/3
.
See auto_assert_raise/3
.
auto_assert_raise(exception, message, function)
View Source (since 0.3.0) (macro)Pattern-generating variant of ExUnit.Assertions.assert_raise/3
.
If the given function does not raise, the assertion will fail.
Like auto_assert/1
, you will be prompted to automatically update
the assertion if the raised raised exception changes.
examples
Examples
You can pass an anonymous function that takes no arguments and is expected to raise an exception.
auto_assert_raise fn ->
some_call_expected_to_raise()
end
# after running the test and accepting changes
auto_assert_raise Some.Exception, fn ->
some_call_expected_to_raise()
end
# optionally include the message
auto_assert_raise Some.Exception, "perhaps with a message", fn ->
some_call_expected_to_raise()
end
A captured function of arity zero can also be used.
auto_assert_raise &some_call_expected_to_raise/0
# after running the test and accepting changes
auto_assert_raise Some.Exception, &some_call_expected_to_raise/0
Pattern-generating variant of ExUnit.Assertions.assert_receive/3
.
timeout
is in milliseconds and defaults to 100
.
examples
Examples
Process.send_after(self(), {:some, :message}, 50)
auto_assert_receive()
# after running the test, messages appearing within 100ms
# will be available as options
auto_assert_receive {:some, :message}
A custom timeout can be specified as a second argument.
Process.send_after(self(), {:some, :message}, 150)
auto_assert_receive nil, 300
# messages appearing within 300ms will now appear as options
auto_assert_receive {:some, :message}, 300
Pattern-generating variant of ExUnit.Assertions.assert_received/2
.
Similar to auto_assert_receive/2
, except that the timeout is set to
0, so the expected message must already be in the current process'
mailbox.
examples
Examples
send(self(), {:some, :message})
auto_assert_received()
# after running the test, messages in the current process
# inbox will be available as options
auto_assert_receive {:some, :message}