PropertyDamage.Stutter (PropertyDamage v0.2.0)

View Source

Stutter testing for idempotency verification.

Stutter testing automatically retries commands to verify that the SUT behaves idempotently - that retrying a command produces the same result.

How It Works

When stutter testing is enabled, some commands are executed multiple times:

  1. First execution: Events applied to projections normally
  2. Retry executions: Events captured but NOT applied to projections
  3. Framework compares retry events to first execution
  4. Mismatch = idempotency violation

This approach tests SUT idempotency without requiring idempotent projections.

Configuration

PropertyDamage.run(
  model: MyModel,
  adapter: MyAdapter,
  stutter: %{
    probability: 0.1,      # 10% of commands stuttered
    max_repeats: 2,        # Up to 2 retries (3 total executions)
    delay_ms: {0, 100},    # Random delay between retries
    commands: :all,        # Or list of specific command modules
    comparison: :strict    # :strict, {:structural, fields}, {:custom, fn}
  }
)

Command Callbacks

Commands can opt into idempotency testing by implementing:

  • idempotent?/0 - Return true if command should be stuttered (default: true)
  • idempotency_key/1 - Return the idempotency key for requests
  • acceptable_retry_events/0 - Event modules acceptable as retry responses

Comparison Modes

  • :strict - Events must be exactly equal
  • {:structural, ignore_fields} - Ignore specified fields when comparing
  • {:custom, fun} - Custom comparison function

Summary

Functions

Build the stutter context passed to adapters during execution.

Compare events from first execution with retry execution.

Default stutter configuration.

Get the idempotency key for a command if it provides one.

Parse stutter configuration from options.

Get the number of retry attempts for a stuttered command.

Get the delay in milliseconds before a retry attempt.

Determine if a command should be stuttered based on configuration.

Functions

build_context(attempt, is_retry, idempotency_key)

@spec build_context(pos_integer(), boolean(), String.t() | nil) :: map()

Build the stutter context passed to adapters during execution.

compare_events(original_events, retry_events, config, command)

@spec compare_events(
  [struct()],
  [struct()],
  PropertyDamage.Stutter.Config.t(),
  struct()
) ::
  :match | {:mismatch, map()}

Compare events from first execution with retry execution.

Returns :match if events are considered equivalent, or {:mismatch, details} if they differ.

default_config()

@spec default_config() :: PropertyDamage.Stutter.Config.t()

Default stutter configuration.

get_idempotency_key(command)

@spec get_idempotency_key(struct()) :: String.t() | nil

Get the idempotency key for a command if it provides one.

parse_config(opts)

@spec parse_config(map() | false | nil) :: PropertyDamage.Stutter.Config.t() | nil

Parse stutter configuration from options.

Accepts either a map of options or false to disable.

retry_count(config)

@spec retry_count(PropertyDamage.Stutter.Config.t()) :: pos_integer()

Get the number of retry attempts for a stuttered command.

Returns a random number between 1 and max_repeats.

retry_delay_ms(config)

@spec retry_delay_ms(PropertyDamage.Stutter.Config.t()) :: non_neg_integer()

Get the delay in milliseconds before a retry attempt.

should_stutter?(command, config)

@spec should_stutter?(
  struct(),
  PropertyDamage.Stutter.Config.t()
) :: boolean()

Determine if a command should be stuttered based on configuration.

The probabilistic check reads the process RNG (:rand). Determinism comes from the executor seeding that RNG once per run (:rand.seed(:exsss, seed) before the run loop), not from any per-call seeding here: re-running with the same seed reproduces the same stutter decisions. The same applies to retry_count/1 and retry_delay_ms/1.