Jump.CredoChecks.VacuousTest (Jump.CredoChecks v0.3.0)

Basics

This check has a base priority of high and works with any version of Elixir.

Explanation

Tests should exercise your application's functions. A test that only asserts on literals, standard library calls, or third-party library calls is not verifying any application behavior.

# ❌ Vacuous — no application code is called
test "example" do
  refute 3 in [1, 2, 5]
  assert byte_size("hello") > 0
  assert :ok == :ok
end

# ✅ Meaningful — exercises application code
test "example" do
  result = MyApp.process("hello")
  assert result == "expected"
end

This is especially useful in the age of LLMs, where AI agents frequently generate low-quality tests with ambitious-sounding names, but whose implementation does not actually guarantee the test title promises.

Check-Specific Parameters

Use the following parameters to configure this check:

:library_modules

Additional library namespaces whose calls should not count as production code, like Jason or Phoenix.

This parameter defaults to [].

:ignore_setup_only_tests?

When true (default), tests that destructure setup context (3-arity test blocks) are considered not vacuous. Set to false to check them too.

This parameter defaults to true.

General Parameters

Like with all checks, general params can be applied.

Parameters can be configured via the .credo.exs config file.