# Design Document

## Overview

This design outlines the comprehensive test infrastructure overhaul for Snakepit using the Supertester framework (github.com/nshkrdotcom/supertester), following the established OTP testing standards documented in docs/code-standards/comprehensive-otp-testing-standards.md.

The current state shows critical gaps including:
- **Minimal test coverage** (only basic module existence test)
- **No OTP-compliant testing patterns** 
- **No usage of proven test helpers** from Supertester
- **No adherence to established code standards**

The solution involves integrating Snakepit with Supertester as a test dependency, implementing all tests using Supertester's OTP helpers, and following the comprehensive testing standards to achieve reliable, maintainable tests.

## Architecture

### High-Level Architecture

```mermaid
graph TB
    subgraph "Supertester Framework (External Dependency)"
        UTH[UnifiedTestFoundation]
        OTP[OTPHelpers]
        GS[GenServerHelpers]
        SUP[SupervisorHelpers]
        MSG[MessageHelpers]
        PERF[PerformanceHelpers]
        CHAOS[ChaosHelpers]
        DATA[DataGenerators]
        ASSERT[Assertions]
    end
    
    subgraph "Snakepit Test Suite"
        TS[Test Support]
        UT[Unit Tests]
        IT[Integration Tests]
        PT[Performance Tests]
        CT[Chaos Tests]
    end
    
    subgraph "Snakepit Components Under Test"
        POOL[Pool]
        WORK[Worker]
        SESS[Session]
        ADAP[Adapter]
        BRID[Bridge]
        REG[Registry]
    end
    
    UTH --> TS
    OTP --> TS
    GS --> TS
    SUP --> TS
    MSG --> TS
    PERF --> TS
    CHAOS --> TS
    DATA --> TS
    ASSERT --> TS
    
    TS --> UT
    TS --> IT
    TS --> PT
    TS --> CT
    
    UT --> POOL
    UT --> WORK
    UT --> SESS
    UT --> ADAP
    IT --> BRID
    IT --> REG
```

### Dependency Integration

```elixir
# In snakepit/mix.exs
defp deps do
  [
    # ... existing deps ...
    {:supertester, github: "nshkrdotcom/supertester", only: :test}
  ]
end
```

### Test Organization Following Code Standards

```
test/
├── support/
│   ├── snakepit_test_helpers.ex    # Snakepit-specific helpers using Supertester
│   ├── mock_adapter.ex              # Mock adapter following OTP patterns
│   └── test_scenarios.ex            # Common test scenarios
├── unit/
│   ├── pool_test.exs               # Using Supertester.OTPHelpers
│   ├── worker_test.exs             # Using Supertester.GenServerHelpers
│   ├── session_test.exs            # Using Supertester isolation
│   ├── adapter_test.exs            # Using Supertester mocking
│   └── bridge_protocol_test.exs    # Using Supertester assertions
├── integration/
│   ├── pool_worker_test.exs        # Using Supertester.SupervisorHelpers
│   ├── session_pool_test.exs       # Using Supertester integration patterns
│   └── registry_test.exs           # Using Supertester.MessageHelpers
├── performance/
│   ├── pool_benchmark_test.exs     # Using Supertester.PerformanceHelpers
│   └── stress_test.exs             # Using Supertester workload patterns
└── chaos/
    ├── worker_failure_test.exs      # Using Supertester.ChaosHelpers
    └── pool_recovery_test.exs       # Using Supertester recovery verification
```

## Components and Interfaces

### 1. Test Support Integration with Supertester

#### Snakepit.TestHelpers
**Purpose**: Snakepit-specific test utilities built on top of Supertester.

```elixir
defmodule Snakepit.TestHelpers do
  @moduledoc """
  Snakepit-specific test helpers built on Supertester framework.
  Follows comprehensive-otp-testing-standards.md patterns.
  """
  
  import Supertester.OTPHelpers
  import Supertester.GenServerHelpers
  import Supertester.DataGenerators
  
  @doc """
  Sets up an isolated Snakepit pool using Supertester patterns.
  No Process.sleep, proper OTP synchronization.
  """
  def with_isolated_pool(opts \\ [], fun) do
    pool_name = unique_process_name("snakepit_pool")
    
    pool_opts = Keyword.merge([
      name: pool_name,
      size: 2,
      adapter: Snakepit.MockAdapter
    ], opts)
    
    {:ok, pool} = setup_isolated_genserver(Snakepit.Pool, "pool_test", pool_opts)
    
    # Wait for pool initialization using OTP patterns
    wait_for_genserver_sync(pool)
    
    try do
      fun.(pool)
    after
      cleanup_on_exit(fn -> GenServer.stop(pool) end)
    end
  end
  
  @doc """
  Creates test adapter configuration using Supertester patterns.
  """
  def test_adapter_config(overrides \\ []) do
    Keyword.merge([
      module: Snakepit.MockAdapter,
      startup_delay: 0,
      command_delay: 0,
      failure_rate: 0
    ], overrides)
  end
  
  @doc """
  Waits for pool to have expected worker count without sleep.
  """
  def wait_for_worker_count(pool, expected_count, timeout \\ 5000) do
    # Use Supertester's wait pattern
    wait_until(fn ->
      {:ok, stats} = call_with_timeout(pool, :get_stats)
      length(stats.workers) == expected_count
    end, timeout)
  end
end
```

#### Snakepit.MockAdapter
**Purpose**: OTP-compliant mock adapter for testing.

```elixir
defmodule Snakepit.MockAdapter do
  @behaviour Snakepit.Adapter
  
  use GenServer
  import Supertester.OTPHelpers
  
  # Adapter behaviour implementation following OTP patterns
  def start_worker(config, opts \\ []) do
    # Use Supertester's isolated process creation
    name = unique_process_name("mock_worker")
    GenServer.start_link(__MODULE__, {config, opts}, name: name)
  end
  
  def execute_command(worker_pid, command, args, opts \\ []) do
    call_with_timeout(worker_pid, {:execute, command, args, opts})
  end
  
  def stop_worker(worker_pid, reason) do
    GenServer.stop(worker_pid, reason)
  end
  
  # GenServer callbacks with test controls
  def init({config, _opts}) do
    state = %{
      config: config,
      responses: %{},
      delays: %{},
      command_history: [],
      failure_injection: nil
    }
    {:ok, state}
  end
  
  def handle_call({:execute, command, args, _opts}, _from, state) do
    # Record command for verification
    state = update_in(state.command_history, &[{command, args, System.monotonic_time()} | &1])
    
    # Apply configured delays without Process.sleep
    if delay = state.delays[command] do
      # Use OTP timer instead of sleep
      Process.send_after(self(), {:delayed_response, command}, delay)
      {:noreply, state}
    else
      response = get_response(state, command, args)
      {:reply, response, state}
    end
  end
  
  # Test control functions
  def set_response(worker_pid, command, response) do
    GenServer.call(worker_pid, {:set_response, command, response})
  end
  
  def inject_failure(worker_pid, failure_type) do
    GenServer.call(worker_pid, {:inject_failure, failure_type})
  end
end
```

### 2. Unit Test Specifications Using Supertester

#### Pool Manager Tests
```elixir
defmodule Snakepit.PoolTest do
  use ExUnit.Case, async: true
  
  import Supertester.OTPHelpers
  import Supertester.GenServerHelpers
  import Supertester.Assertions
  import Snakepit.TestHelpers
  
  describe "pool initialization" do
    test "starts with configured number of workers" do
      with_isolated_pool([size: 3], fn pool ->
        # No Process.sleep needed
        wait_for_worker_count(pool, 3)
        assert_genserver_state(pool, fn state ->
          length(state.workers) == 3
        end)
      end)
    end
    
    test "handles concurrent worker initialization" do
      # Supertester handles concurrent testing properly
      operations = for i <- 1..5, do: {:call, :get_stats}
      
      with_isolated_pool([size: 5], fn pool ->
        {:ok, results} = concurrent_calls(pool, operations, 5)
        assert Enum.all?(results, &match?({:ok, _}, &1))
      end)
    end
    
    test "respects startup timeout configuration" do
      slow_adapter = test_adapter_config(startup_delay: 100)
      
      assert_raise RuntimeError, ~r/startup timeout/, fn ->
        with_isolated_pool([adapter_config: slow_adapter, startup_timeout: 50], fn _pool ->
          :ok
        end)
      end
    end
  end
  
  describe "request handling with OTP patterns" do
    setup do
      {:ok, pool} = setup_isolated_genserver(Snakepit.Pool, "request_test", size: 2)
      %{pool: pool}
    end
    
    test "distributes requests to available workers", %{pool: pool} do
      # Use Supertester's stress testing
      {:ok, stats} = stress_test_server(pool, [
        {:call, {:execute, "ping", %{}, []}},
        {:call, {:execute, "ping", %{}, []}},
        {:call, {:execute, "ping", %{}, []}}
      ], 5000)
      
      assert stats.completed == 3
      assert stats.errors == 0
    end
    
    test "queues requests when workers busy" do
      # Test without Process.sleep using Supertester patterns
      with_isolated_pool([size: 1], fn pool ->
        # Make worker busy
        task1 = Task.async(fn ->
          call_with_timeout(pool, {:execute, "slow", %{}, []}, 5000)
        end)
        
        # Ensure first request is processing
        wait_for_genserver_sync(pool)
        
        # Second request should queue
        task2 = Task.async(fn ->
          call_with_timeout(pool, {:execute, "fast", %{}, []}, 5000)
        end)
        
        # Verify queueing using OTP patterns
        assert_genserver_state(pool, fn state ->
          :queue.len(state.request_queue) > 0
        end)
        
        Task.await_many([task1, task2])
      end)
    end
  end
  
  describe "worker management following OTP standards" do
    test "tracks worker state accurately" do
      import Supertester.SupervisorHelpers
      
      with_isolated_pool([size: 3], fn pool ->
        # Get worker PIDs safely
        {:ok, stats} = get_server_state_safely(pool)
        worker_pids = stats.workers
        
        # Verify all workers alive
        Enum.each(worker_pids, &assert_process_alive/1)
        
        # Kill a worker
        [worker | _] = worker_pids
        Process.exit(worker, :kill)
        
        # Wait for supervisor recovery without sleep
        wait_for_process_death(worker)
        wait_for_supervisor_restart(pool)
        
        # Verify pool recovered
        assert_genserver_responsive(pool)
        wait_for_worker_count(pool, 3)
      end)
    end
  end
end
```

#### Session Management Tests
```elixir
defmodule Snakepit.SessionTest do
  use ExUnit.Case, async: true
  
  import Supertester.OTPHelpers
  import Supertester.DataGenerators
  import Supertester.Assertions
  import Snakepit.TestHelpers
  
  describe "session affinity with OTP patterns" do
    test "maintains worker affinity across requests" do
      session_id = unique_session_id("test")
      
      with_isolated_pool([size: 3], fn pool ->
        # First request establishes affinity
        {:ok, result1} = Snakepit.execute_in_session(session_id, "get_pid", %{})
        worker_pid1 = result1.pid
        
        # Subsequent requests use same worker
        for _ <- 1..5 do
          {:ok, result} = Snakepit.execute_in_session(session_id, "get_pid", %{})
          assert result.pid == worker_pid1
        end
        
        # Different session uses different worker
        other_session = unique_session_id("other")
        {:ok, result2} = Snakepit.execute_in_session(other_session, "get_pid", %{})
        
        # May get same worker with small pool, but verify session tracking
        assert_genserver_state(pool, fn state ->
          Map.has_key?(state.sessions, session_id) and
          Map.has_key?(state.sessions, other_session)
        end)
      end)
    end
    
    test "handles concurrent session operations" do
      # Use Supertester's concurrent testing
      session_ids = for i <- 1..10, do: unique_session_id("concurrent_#{i}")
      
      with_isolated_pool([size: 5], fn pool ->
        # Concurrent session creation
        tasks = for session_id <- session_ids do
          Task.async(fn ->
            Snakepit.execute_in_session(session_id, "init", %{})
          end)
        end
        
        results = Task.await_many(tasks, 5000)
        assert Enum.all?(results, &match?({:ok, _}, &1))
        
        # Verify all sessions tracked
        assert_genserver_state(pool, fn state ->
          Enum.all?(session_ids, &Map.has_key?(state.sessions, &1))
        end)
      end)
    end
  end
end
```

### 3. Integration Tests Using Supertester

```elixir
defmodule Snakepit.PoolWorkerIntegrationTest do
  use ExUnit.Case, async: true
  
  import Supertester.SupervisorHelpers
  import Supertester.OTPHelpers
  import Supertester.Assertions
  
  describe "pool-worker supervision integration" do
    test "pool supervisor restarts failed workers" do
      {:ok, supervisor} = setup_isolated_supervisor(
        Snakepit.WorkerSupervisor,
        "integration_test"
      )
      
      # Start pool under supervisor
      {:ok, pool} = Supervisor.start_child(supervisor, {
        Snakepit.Pool,
        [name: unique_process_name("supervised_pool"), size: 3]
      })
      
      # Get initial worker
      [worker | _] = get_child_pids(supervisor)
      
      # Test restart strategy
      test_child_restart_strategies(supervisor, [
        {:kill_child, worker, :one_for_one}
      ])
      
      # Verify pool still functional
      assert_genserver_responsive(pool)
      assert_all_children_alive(supervisor)
    end
  end
end
```

### 4. Performance Tests Using Supertester

```elixir
defmodule Snakepit.PoolBenchmarkTest do
  use ExUnit.Case, async: false
  
  import Supertester.PerformanceHelpers
  import Snakepit.TestHelpers
  
  @tag :benchmark
  describe "pool performance benchmarks" do
    test "measures throughput with varying worker counts" do
      for worker_count <- [1, 2, 4, 8, 16] do
        with_isolated_pool([size: worker_count], fn pool ->
          result = benchmark_operations([
            {"simple command", fn -> 
              Snakepit.execute("echo", %{msg: "test"}, pool: pool)
            end}
          ], 1000)
          
          assert_performance_within_bounds(result, %{
            max_time: 5000,
            min_throughput: worker_count * 100  # ops/sec
          })
        end)
      end
    end
    
    test "workload pattern testing" do
      with_isolated_pool([size: 4], fn pool ->
        # Test different workload patterns from Supertester
        for pattern <- [:steady, :burst, :ramp_up] do
          result = workload_pattern_test(
            fn -> Snakepit.execute("work", %{}, pool: pool) end,
            pattern
          )
          
          # No regression from baseline
          assert_no_performance_regression(result, get_baseline(pattern))
        end
      end)
    end
  end
end
```

### 5. Chaos Tests Using Supertester

```elixir
defmodule Snakepit.WorkerFailureChaosTest do
  use ExUnit.Case, async: true
  
  import Supertester.ChaosHelpers
  import Supertester.Assertions
  import Snakepit.TestHelpers
  
  @tag :chaos
  describe "worker failure resilience" do
    test "recovers from random worker crashes" do
      with_isolated_pool([size: 5], fn pool ->
        # Get initial state
        {:ok, initial_state} = GenServer.call(pool, :get_stats)
        
        # Inject random failures
        chaos_scenario = [
          {:random_process_kill, 2, 1000},
          {:memory_pressure, 50, 2000},
          {:cpu_spike, 90, 1000}
        ]
        
        # Run chaos test
        {:ok, chaos_result} = chaos_test_orchestrator(
          chaos_scenario,
          :sequential,
          target: pool
        )
        
        # Verify recovery
        {:ok, recovery_verified} = verify_system_recovery(
          initial_state,
          timeout: 10_000
        )
        
        assert recovery_verified
        assert_genserver_responsive(pool)
        assert_resilience_to_chaos(pool, chaos_scenario)
      end)
    end
  end
end
```

## Test Execution Strategy

### Following Code Standards

All tests must follow docs/code-standards/comprehensive-otp-testing-standards.md:

1. **Zero Process.sleep** - Use Supertester synchronization helpers
2. **Isolated processes** - Use Supertester's unique naming
3. **Proper monitoring** - Use Supertester's lifecycle helpers
4. **OTP patterns** - Follow Supertester's established patterns

### CI/CD Integration

```yaml
# .github/workflows/test.yml
test:
  steps:
    - name: Run tests with Supertester
      run: |
        mix deps.get
        mix test --cover
        mix test.pattern_check  # Verify no Process.sleep
        mix test.isolation_check # Verify proper isolation
```

### Quality Gates

- All tests must use `async: true` with Supertester isolation
- Zero Process.sleep usage (enforced by pattern check)
- >90% code coverage
- All Supertester assertions must pass
- Follow comprehensive-otp-testing-standards.md

## Success Metrics

### Quantitative Metrics
- **Test coverage** >90% using Supertester patterns
- **Test execution** <30 seconds with async: true
- **Zero Process.sleep** usage verified by tooling
- **All tests use Supertester** helpers and assertions
- **Performance baselines** established with PerformanceHelpers

### Qualitative Metrics
- **Consistent with code standards** in comprehensive-otp-testing-standards.md
- **Educational value** demonstrating proper Supertester usage
- **Maintainable tests** following established patterns
- **Clear diagnostics** from Supertester assertions
- **Reliable CI/CD** with no flaky tests