PropertyDamage.Mutation (PropertyDamage v0.2.0)

View Source

Mutation testing for PropertyDamage.

Mutation testing verifies that your property-based tests are effective by:

  1. Injecting faults (mutations) into adapter responses
  2. Running tests against the mutated responses
  3. Checking if tests catch (kill) the mutations

A killed mutant means your tests detected the simulated bug (good). A survived mutant means your tests missed the bug (bad - weak tests).

Usage

# Run mutation testing
{:ok, report} = PropertyDamage.Mutation.run(
  model: MyModel,
  adapter: MyAdapter,
  adapter_config: %{base_url: "http://localhost:4000"},
  target_score: 0.80
)

# Check if tests are effective
if PropertyDamage.Mutation.passes?(report) do
  IO.puts("Tests are effective!")
else
  IO.puts("Tests need improvement")
  IO.puts(PropertyDamage.Mutation.format(report))
end

# Get detailed analysis
analysis = PropertyDamage.Mutation.analyze(report)
IO.puts(PropertyDamage.Mutation.Analysis.format(analysis))

Mutation Operators

  • :value - Mutates numeric and string values (zero, negate, off-by-one)
  • :omission - Removes fields from events
  • :status - Changes success/error outcomes
  • :event - Modifies event contents and structure
  • :boundary - Pushes values to edge cases (0, -1, max, nil)

Options

  • :model - The model module (required)
  • :adapter - The adapter module (required)
  • :adapter_config - Configuration for the adapter
  • :operators - List of operator names (default: all)
  • :mutations_per_command - Max mutations per command (default: 5)
  • :max_runs - Property test runs per mutation (default: 10)
  • :target_score - Target mutation score (default: 0.80)
  • :timeout_ms - Timeout per mutation test (default: 30000)
  • :verbose - Print progress (default: false)

Summary

Functions

analyze(report)

Analyzes a mutation report to identify weaknesses.

available_operators()

Returns all available mutation operator names.

format(report, format \\ :terminal)

Formats a mutation report for display.

passes?(report)

Checks if a mutation report passes the target score.

run(opts)

Runs mutation testing against a model.

weakest_commands(report)

Returns the weakest commands from a report, sorted by kill rate.

weakest_operators(report)

Returns the weakest operators from a report, sorted by kill rate.

Functions

analyze(report)

@spec analyze(PropertyDamage.Mutation.Report.t()) ::
  PropertyDamage.Mutation.Analysis.analysis()

Analyzes a mutation report to identify weaknesses.

Returns insights about:

  • Weak commands (low kill rates)
  • Weak operators (types of mutations that survive)
  • Unchecked fields
  • Actionable suggestions

Examples

analysis = PropertyDamage.Mutation.analyze(report)
IO.puts(analysis.summary)
Enum.each(analysis.suggestions, &IO.puts/1)

available_operators()

@spec available_operators() :: [atom()]

Returns all available mutation operator names.

Examples

PropertyDamage.Mutation.available_operators()
# => [:value, :omission, :status, :event, :boundary]

format(report, format \\ :terminal)

@spec format(PropertyDamage.Mutation.Report.t(), atom()) :: String.t()

Formats a mutation report for display.

Formats

  • :terminal - ASCII boxes for console output (default)
  • :markdown - Markdown tables for documentation
  • :json - JSON for programmatic analysis

Examples

IO.puts(PropertyDamage.Mutation.format(report))
IO.puts(PropertyDamage.Mutation.format(report, :markdown))

passes?(report)

@spec passes?(PropertyDamage.Mutation.Report.t()) :: boolean()

Checks if a mutation report passes the target score.

Examples

if PropertyDamage.Mutation.passes?(report) do
  IO.puts("Tests are effective!")
end

run(opts)

@spec run(keyword()) :: {:ok, PropertyDamage.Mutation.Report.t()} | {:error, term()}

Runs mutation testing against a model.

Returns {:ok, report} with mutation testing results.

Examples

{:ok, report} = PropertyDamage.Mutation.run(
  model: MyModel,
  adapter: MyAdapter,
  operators: [:value, :omission],
  target_score: 0.80
)

report.mutation_score  # => 0.85
report.killed          # => 17
report.survived        # => 3

weakest_commands(report)

@spec weakest_commands(PropertyDamage.Mutation.Report.t()) :: [
  {module(), PropertyDamage.Mutation.Report.command_stats()}
]

Returns the weakest commands from a report, sorted by kill rate.

Examples

for {cmd, stats} <- PropertyDamage.Mutation.weakest_commands(report) do
  IO.puts("#{cmd}: #{stats.score * 100}%")
end

weakest_operators(report)

@spec weakest_operators(PropertyDamage.Mutation.Report.t()) :: [
  {atom(), PropertyDamage.Mutation.Report.command_stats()}
]

Returns the weakest operators from a report, sorted by kill rate.