ForgeCredoChecks.NoSourceInspectionInTest (forge_credo_checks v0.7.0)

Basics

This check has a base priority of high and works with any version of Elixir.

Explanation

Tests must exercise behaviour through the public API, not inspect the TEXT or AST of production source files.

Why

A "contract test" that File.read!s a lib/*.ex module and asserts on its source with =~ / Regex / Code.string_to_quoted is brittle. It breaks on behaviour-preserving refactors (a case becoming multi-clause function heads, a formatter reflow, a rename) even though the runtime behaviour is unchanged, and it gives false confidence: a matched substring may be a comment or a shadowed clause, and a passing grep never actually ran the code. AST inspection of the same source (Code.string_to_quoted + Macro walking) has the same fault: it asserts on structure, not behaviour.

Test the real contract instead. Call the producer to get each output shape, call the consumer with each shape, and assert the returned value or effect. If the code under test is private, make it public (a @doc false function is fine) or extract a reactor closure into a named public function. Exposing a function to test critical behaviour is always preferable to asserting on source text.

Bad

# A literal source read.
@impl_source "lib/forge_symphony/.../implementation.ex"
source = File.read!(@impl_source)
assert source =~ "%StepFailure{}"

# Source paths carried as DATA, then read + parsed dynamically. Same
# anti-pattern, one indirection removed.
@registry [%{id: :x, file: "lib/forge_symphony/.../step.ex"}]
defp source_ast(rel), do: rel |> File.read!() |> Code.string_to_quoted!()

Good

assert {:ok, %{status: :failed, error: :boom}} =
         Implementation.validate_dispatch_result(%StepFailure{error: :boom}, ...)

What is flagged

Within test files (included_paths, default [~r"_test\.exs$"]):

• a module attribute whose value IS a lib/*.ex path (@impl_source "lib/.../x.ex");
• File.read! / File.read / File.stream! with a literal lib/*.ex argument;
• Code.string_to_quoted (and the !, _with_comments, eval_string, eval_file variants) on a literal lib/*.ex path, OR anywhere in a file that also carries lib/*.ex paths as data -- the dynamic-path form that reads a table of source files and parses each.

Carrying a lib/*.ex path as plain data (a routing-decision fixture, an ADR-rules JSON blob) is NOT flagged on its own: only reading or parsing production source is. Reading generated or fixture files (paths that do not point at lib/*.ex) is not flagged, and parsing that only ever touches non-lib source (a meta-lint over the test tree) is left alone.

Check-Specific Parameters

Use the following parameters to configure this check:

:included_paths

Paths (regexes or substrings) the check runs on (default: *_test.exs).

This parameter defaults to [~r/_test\.exs$/].

General Parameters

Like with all checks, general params can be applied.

Parameters can be configured via the .credo.exs config file.