๐งช Eliot Testing Guide ๐งช
View SourceWelcome to the Eliot testing documentation! This guide explains how to run tests, understand the test suite, and create new tests for the Eliot system.
๐ Table of Contents
- Test Suite Overview
- Running Tests
- Understanding Test Types
- Test Framework (ExUnit)
- Writing New Tests
- Test Coverage
- Continuous Integration
- Testing Best Practices
Test Suite Overview
Eliot includes a comprehensive test suite that validates:
- Core Functionality: Correctness of module logic and public APIs.
- OTP Behavior: Proper GenServer/Supervisor initialization, state changes, and shutdown.
- Error Handling: Circuit breaker and retry logic correctness.
- Integration: How different components of the system interact.
Tests are organized into two main categories:
- Unit Tests: Testing individual modules and functions in isolation.
- Integration Tests: Testing the interaction between different components (e.g.,
ErrorHandler
andLogger
).
Running Tests
Running All Tests
The simplest way to run all tests is using the standard mix
command:
mix test
This will compile the project if needed and run all tests found in the test/
directory.
Running a Single Test File
To run tests from a single file:
mix test test/eliot/error_handler_test.exs
Running a Single Test
To run a specific test case from a file, you can specify the line number:
mix test test/eliot/error_handler_test.exs:25
Excluding or Including Tests with Tags
You can tag tests to run specific categories. For example, integration tests are tagged with :integration
.
# Run only integration tests
mix test --only integration
# Run all tests except integration tests
mix test --exclude integration
Understanding Test Types
Unit Tests
Unit tests check individual modules and functions in isolation. These tests focus on:
- Public function return values for given inputs.
- GenServer callbacks and state transitions.
- Logic inside private functions (tested via their public-facing callers).
Sample unit test files:
test/eliot/elixir_test.exs
test/eliot/error_handler_test.exs
test/eliot/logger_test.exs
Integration Tests
Integration tests verify that different components work together correctly. For Eliot, this includes:
- Ensuring an error handled by
ErrorHandler
is correctly logged byEliot.Logger
. - Verifying the
Eliot.Application
correctly starts and supervises its children.
Sample integration test files:
test/eliot/integration/mqtt_integration_test.exs
Test Framework (ExUnit)
Eliot uses Elixir's built-in testing framework, ExUnit.
Test Structure
Tests are defined inside modules using use ExUnit.Case
. The describe
block groups related tests, and each test
block defines a single test case.
defmodule Eliot.MyModuleTest do
use ExUnit.Case, async: true
describe "my_function/2" do
test "does something correctly" do
assert Eliot.MyModule.my_function(:a, :b) == :ok
end
end
end
Setup
You can use setup
or setup_all
blocks to run setup code before tests. setup
runs before each test in a block, while setup_all
runs once for the entire block.
defmodule Eliot.MyModuleTest do
use ExUnit.Case, async: true
setup do
# This runs before each test
{:ok, pid} = MyGenServer.start_link()
# The returned value is passed to the test context
%{pid: pid}
end
test "sends a message to the genserver", context do
assert :ok == MyGenServer.do_something(context.pid)
end
end
Assertions
ExUnit provides various assertion macros:
assert some_condition == true
refute some_condition == true
assert_raise(MyError, fn -> ... end)
assert_receive(:my_message, 100)
assert_received(:my_message)
Mocking
For isolating components from their dependencies, we recommend using a library like Mox. Mox allows you to define mock implementations of your modules' behaviors for testing.
Writing New Tests
To create a new unit test:
- Create a new file in the
test/eliot/
directory, ending with_test.exs
. - Follow this basic structure:
defmodule Eliot.NewFeatureTest do use ExUnit.Case, async: true describe "new_feature_function/1" do test "handles valid input correctly" do # Given valid input input = "valid" # When the function is called result = Eliot.NewFeature.new_feature_function(input) # Then the result is correct assert result == :ok end test "handles invalid input with an error tuple" do assert Eliot.NewFeature.new_feature_function(nil) == {:error, :invalid_input} end end end
- Run
mix test
to ensure your new tests are picked up and pass.
Test Coverage
We aim for high code coverage across the codebase.
Measuring Coverage
You can generate a test coverage report by running:
mix test --cover
This generates a report in cover/index.html
which you can open in your browser to see which lines of code are covered by tests.
Coverage Requirements
- Core functionality in
Eliot
,Eliot.ErrorHandler
, andEliot.Logger
should have >95% coverage. - All public API functions must have tests.
- Error handling paths must be tested explicitly.
Continuous Integration
Eliot uses GitHub Actions for continuous integration. The workflow runs automatically on pull requests and includes:
- Running
mix test
- Checking formatting with
mix format --check-formatted
- Running static analysis with
mix credo
Testing Best Practices
- Test One Thing at a Time: Each test should verify a single, specific behavior.
- Use Descriptive Names: Test names should clearly describe what is being tested.
- Isolate Tests: Use
async: true
where possible to run tests concurrently and ensure they don't depend on shared state. - Test the Public API: Focus tests on the public-facing functions of your modules, not the private implementation details.
- Test Failure Cases: Verify that your functions fail appropriately with invalid inputs and that your application recovers gracefully from errors.
If you have questions about testing, please reach out via GitHub issues.