PropertyDamage.Integration (PropertyDamage v0.2.0)

View Source

Integration testing utilities for running PropertyDamage against live services.

This module provides helpers for running property-based tests against real systems, with support for:

  • Health checks before testing
  • Database reset between runs
  • Report generation (terminal, markdown, JUnit XML)
  • Bug hunting mode (run until N unique bugs found)
  • Chaos testing with Toxiproxy

Usage

# Basic integration test
{:ok, result} = PropertyDamage.Integration.run(
  model: MyModel,
  adapter: MyAdapter,
  adapter_config: %{base_url: "http://localhost:4000"},
  max_runs: 100
)

# Bug hunting mode
{:ok, bugs} = PropertyDamage.Integration.hunt_bugs(
  model: MyModel,
  adapter: MyAdapter,
  adapter_config: %{base_url: "http://localhost:4000"},
  stop_after: 10  # Stop after 10 unique bugs
)

Health Checks

Before running tests, the integration runner can verify the service is healthy:

PropertyDamage.Integration.run(
  # ...
  health_check: %{
    url: "http://localhost:4000/api/health",
    timeout_ms: 5000,
    retries: 10
  }
)

Reports

Generate reports in various formats:

PropertyDamage.Integration.run(
  # ...
  report: %{
    format: :junit,
    path: "reports/integration.xml"
  }
)

Supported formats: :terminal, :markdown, :junit, :json

Summary

Functions

Generate a test report from integration results.

Perform a health check against a service.

Run tests until a specified number of unique bugs are found.

Run integration tests against a live service.

Functions

generate_report(result, opts)

@spec generate_report(map(), keyword() | map()) :: :ok

Generate a test report from integration results.

Formats

  • :terminal - Print to console
  • :markdown - Generate markdown file
  • :junit - Generate JUnit XML for CI
  • :json - Generate JSON report

Options

  • :format - Report format (required)
  • :path - Output file path (required for file formats)

health_check(opts)

@spec health_check(keyword()) :: :ok | {:error, term()}

Perform a health check against a service.

Options

  • :url - Health check URL (required)
  • :timeout_ms - Total timeout (default: 30000)
  • :retries - Number of retries (default: 30)
  • :interval_ms - Interval between retries (default: 1000)

Returns

  • :ok - Service is healthy
  • {:error, reason} - Health check failed

hunt_bugs(opts)

@spec hunt_bugs(keyword()) :: {:ok, [map()]}

Run tests until a specified number of unique bugs are found.

Options

  • :model - The model module (required)
  • :adapter - The adapter module (required)
  • :adapter_config - Configuration for the adapter (required)
  • :stop_after - Stop after finding this many unique bugs (default: 10)
  • :max_runs - Maximum runs before giving up (default: :unlimited)
  • :save_to - Directory to save discovered bugs (optional)
  • :verbose - Print progress (default: true)

Returns

  • {:ok, bugs} - List of unique bugs found

run(opts)

@spec run(keyword()) :: {:ok, map()} | {:error, map()}

Run integration tests against a live service.

Options

  • :model - The model module (required)
  • :adapter - The adapter module (required)
  • :adapter_config - Configuration for the adapter (required)
  • :max_runs - Number of test runs (default: 100)
  • :max_commands - Max commands per run (default: 50)
  • :health_check - Health check configuration (optional)
  • :reset_fn - Function to reset state between runs (optional)
  • :report - Report configuration (optional)
  • :verbose - Print progress (default: true)
  • :stop_on_failure - Stop on first failure (default: false)
  • :save_failures - Directory to save failures (optional)

Returns

  • {:ok, result} - All tests passed
  • {:error, result} - Tests failed with failure details