defmodule Mneme do @external_resource "mix.exs" @moduledoc """ /ni:mi:/ - Snapshot testing integrated into ExUnit. Snapshot tests assert that some expression matches a reference value. It's like an ExUnit `assert`, except that the reference value is managed for you by Mneme. Mneme follows in the footsteps of existing snapshot testing libraries like [Insta](https://insta.rs/) (Rust), [expect-test](https://github.com/janestreet/ppx_expect) (OCaml), and [assert_value](https://github.com/assert-value/assert_value_elixir) (Elixir). Instead of simple value or string comparison, however, Mneme leans heavily into pattern matching. ## Example Let's say you've written a test for a function that removes even numbers from a list: test "drop_evens/1 should remove all even numbers from an enum" do auto_assert drop_evens(1..10) auto_assert drop_evens([]) auto_assert drop_evens([:a, :b, 2, :c]) end The first time you run this test, you'll see interactive prompts for each call to `auto_assert` showing a diff and asking if you'd like to accept the generated pattern. After accepting them, your test is updated: test "drop_evens/1 should remove all even numbers from an enum" do auto_assert [1, 3, 5, 7, 9] <- drop_evens(1..10) auto_assert [] <- drop_evens([]) auto_assert [:a, :b, :c] <- drop_evens([:a, :b, 2, :c]) end The next time you run this test, you won't receive a prompt and these will act (almost) like any other assertion. If the result of the call ever changes, you'll be prompted again and can choose to update the test or reject it and let it fail. With a few exceptions, `auto_assert/1` acts very similarly to a normal `assert`. See the [macro docs](`auto_assert/1`) for a list of differences. ## Quick start 1. Add `:mneme` do your deps in `mix.exs`: defp deps do [ {:mneme, "~> #{Mneme.MixProject.version()}", only: :test} ] end 2. Add `:mneme` to your `:import_deps` in `.formatter.exs`: [ import_deps: [:mneme], inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"] ] 3. Start Mneme right after you start ExUnit in `test/test_helper.exs`: ExUnit.start() Mneme.start() 4. Add `use Mneme` wherever you `use ExUnit.Case`: defmodule MyTest do use ExUnit.Case, async: true use Mneme test "arithmetic" do # use auto_assert instead of ExUnit's assert - run this test # and delight in all the typing you don't have to do auto_assert 2 + 2 end end ## Match patterns Mneme tries to generate match patterns that are equivalent to what a human (or at least a nice LLM) would write. Basic data types like strings, numbers, lists, tuples, etc. will be as you would expect. Some values, however, do not have a literal representation that can be used in a pattern match. Pids are such an example. For those, guards are used: auto_assert self() # after running the test and accepting the change auto_assert pid when is_pid(pid) <- self() Additionally, local variables can be found and pinned as a part of the pattern. This keeps the number of hard-coded values down, reducing the likelihood that tests have to be updated in the future. test "create_post/1 creates a new post with valid attrs", %{user: user} do valid_attrs = %{title: "my_post", author: user} auto_assert create_post(valid_attrs) end # after running the test test "create_post/1 creates a new post with valid attrs", %{user: user} do valid_attrs = %{title: "my_post", author: user} auto_assert {:ok, %Post{title: "my_post", author: ^user}} <- create_post(valid_attrs) end In many cases, multiple valid patterns will be possible. Usually, the "simplest" pattern will be selected by default when you are prompted, but you can cycle through the options as well. ### Non-exhaustive list of special cases * Pinned variables are generated by default if a value is equal to a variable in scope. * Date and time values are written using their sigil representation. * Struct patterns only include fields that are different from the struct defaults. * Structs defined by Ecto schemas exclude primary keys, association foreign keys, and auto generated fields like `:inserted_at` and `:updated_at`. This is because these fields are often randomly generated and would fail on subsequent tests. ## Configuration There are a few controls that can be used to change Mneme's behavior when it runs auto-assertions. These can be set at the module-level by passing options to `use Mneme`, the `describe` level using the `@mneme_describe` attribute, or the `test` level using the `@mneme` attribute. For instance: defmodule MyTest do use ExUnit.Case # reject all changes to auto-assertions by default use Mneme, action: :reject test "this test will fail" do auto_assert 1 + 1 end describe "some describe block" do # accept all changes to auto-assertions in this describe block @mneme_describe action: :accept test "this will update without prompting" do auto_assert 2 + 2 end # prompt for any changes in this test @mneme action: :prompt test "this will prompt before updating" do auto_assert 3 + 3 end end end ### Options #{Mneme.Options.docs()} ## Formatting Mneme uses [`Rewrite`](https://github.com/hrzndhrn/rewrite) to update source source code, formatting that code before saving the file. Currently, the Elixir formatter and `FreedomFormatter` are supported. **If you do not use a formatter, the first auto-assertion will reformat the entire file.** ## Continuous Integration In a CI environment, Mneme will not attempt to prompt and update any assertions. This behavior is enabled by the `CI` environment variable, which is set by convention by many continuous integration providers. ```bash export CI=true ``` ## Editor support Guides for optional editor integration can be found here: * [`VS Code`](vscode_setup.md) ## Acknowledgements Special thanks to: * [_What if writing tests was a joyful experience?_](https://blog.janestreet.com/the-joy-of-expect-tests/), from the Jane Street Tech Blog, for inspiring this library. * [Sourceror](https://github.com/doorgan/sourceror), a library that makes complex code modifications simple. * [Rewrite](https://github.com/hrzndhrn/rewrite), which provides the diff functionality present in Mneme. * [Owl](https://github.com/fuelen/owl), which makes it much easier to build a pretty CLI. * [Insta](https://insta.rs/), a snapshot testing tool for Rust, whose great documentation provided an excellent reference for snapshot testing. * [assert_value](https://github.com/assert-value/assert_value_elixir), an existing Elixir project that provides similar functionality. Thank you for paving the way! """ @doc """ Sets up Mneme configuration for this module and imports `auto_assert/1`. This macro accepts all options described in the "Configuration" section above. ## Example defmodule MyTest do use ExUnit.Case use Mneme # <- add this test "..." do auto_assert ... end end """ defmacro __using__(opts) do quote do import Mneme, only: [auto_assert: 1] require Mneme.Options Mneme.Options.register_attributes(unquote(opts)) end end @doc """ Starts Mneme to run auto-assertions as they appear in your tests. This will almost always be added to your `test/test_helper.exs`, just below the call to `ExUnit.start()`: # test/test_helper.exs ExUnit.start() Mneme.start() """ def start do ExUnit.configure( formatters: [Mneme.ExUnitFormatter], default_formatter: ExUnit.CLIFormatter, timeout: :infinity ) children = [ Mneme.Server ] Supervisor.start_link(children, strategy: :one_for_one) end @doc """ Generate or run an assertion. `auto_assert` generates assertions when tests run, issuing a terminal prompt before making any changes (unless configured otherwise). auto_assert [1, 2] ++ [3, 4] # after running the test and accepting the change auto_assert [1, 2, 3, 4] <- [1, 2] ++ [3, 4] If the match no longer succeeds, a warning and new prompt will be issued to update it to the new value. auto_assert [1, 2, 3, 4] <- [1, 2] ++ [:a, :b] # after running the test and accepting the change auto_assert [1, 2, :a, :b] <- [1, 2] ++ [:a, :b] Prompts are only issued if the pattern doesn't match the value, so that pattern can also be changed manually. # this assertion succeeds, so no prompt is issued auto_assert [1, 2, | _] <- [1, 2] ++ [:a, :b] ## Differences from ExUnit `assert` The `auto_assert` macro is meant to match `assert` as closely as possible. In fact, it generates ExUnit assertions under the hood. There are, however, a few small differences to note: * Pattern-matching assertions use the `<-` operator instead of the `=` match operator. Value-comparison assertions still use `==` (for instance, when the expression returns `nil` or `false`). * Guards can be added with a `when` clause, while `assert` would require a second assertion. For example: auto_assert pid when is_pid(pid) <- self() assert pid = self() assert is_pid(pid) * Bindings in an `auto_assert` are not available outside of that assertion. For example: auto_assert pid when is_pid(pid) <- self() pid # ERROR: pid is not bound If you need to use the result of the assertion, it will evaluate to the expression's value. pid = auto_assert pid when is_pid(pid) <- self() pid # pid is the result of self() """ defmacro auto_assert(body) do code = {:auto_assert, Macro.Env.location(__CALLER__), [body]} Mneme.Assertion.build(code, __CALLER__) end end