PropertyDamage.Model.Projection.Statistics (PropertyDamage v0.2.0)
View SourceProjection that computes statistical properties over event streams.
Traditional assertions check exact conditions. Statistical projections enable probabilistic assertions like:
- "p99 latency < 100ms"
- "error rate < 1%"
- "success rate > 99% over last 100 operations"
Why Statistical Projections?
Nemesis can inject latency, but we need statistical assertions to verify the system handles it gracefully:
- A single slow request isn't a bug
- 50% of requests being slow IS a bug
- 1% error rate might be acceptable; 10% is not
Usage
defmodule MyModel do
def assertion_projections do
[
{PropertyDamage.Model.Projection.Statistics, [
window_size: 100,
assertions: [
{:p99_latency_ms, :less_than, 500},
{:error_rate, :less_than, 0.05},
{:success_rate, :greater_than, 0.95}
]
]}
]
end
endTracked Metrics
:p50_latency_ms- Median latency:p95_latency_ms- 95th percentile latency:p99_latency_ms- 99th percentile latency:max_latency_ms- Maximum observed latency:mean_latency_ms- Average latency:success_rate- Ratio of successes to total:error_rate- Ratio of errors to total:throughput- Operations per second (requires time tracking)
Recording Metrics
Events should include latency information. The projection looks for:
:latency_msor:duration_msfields for latency tracking- Success events vs error events for rate calculation
You can also record metrics explicitly in your adapter.
Limitations
- Statistics only meaningful with sufficient sample size
- Shrinking statistical failures is problematic (minimal case may not reproduce)
- Thresholds are environment-dependent (CI vs production hardware)
Summary
Functions
Check all configured assertions against current metrics.
Compute all metrics from current state.
Format statistics as a human-readable string.
Record an error/failure.
Record a latency sample.
Record a successful operation.
Get a summary of current statistics.
Types
@type assertion() :: {metric(), comparator(), number()}
@type comparator() :: :less_than | :greater_than | :equal_to
@type metric() ::
:p50_latency_ms
| :p95_latency_ms
| :p99_latency_ms
| :max_latency_ms
| :mean_latency_ms
| :success_rate
| :error_rate
| :total_count
@type t() :: %PropertyDamage.Model.Projection.Statistics{ assertions: [assertion()], current_step: non_neg_integer(), error_count: non_neg_integer(), latency_samples: :queue.queue(float()), start_time: integer(), success_count: non_neg_integer(), window_size: pos_integer() }
Functions
Check all configured assertions against current metrics.
Compute all metrics from current state.
Format statistics as a human-readable string.
Record an error/failure.
Record a latency sample.
Record a successful operation.
Get a summary of current statistics.