ExQuality.Scope (ExQuality v0.13.0)

Resolves how much code a stage runs over, as opposed to which stages run.

Every skip switch this library has narrows stages. An agent iterating on one file does not want fewer checks, it wants the same checks over less code: on a 3,700-test umbrella the suite is 68% of a run's wall clock, and a one-file change needs a handful of test files rather than all of them.

A scope is one of:

:all - everything, which is what an unscoped run has always done
:changed - the test files that map to the files changed against a base ref
a glob string - the test files matching it

The one failure mode that matters

A scoped run that resolves to no files must never report green, because it fails in the safe-looking direction: mix quality exits 0, the report says "status": "ok", and nothing ran. So an empty resolution is not an empty run, it falls back to the full suite and says why in fallback_reason.

For the same reason resolve/2 reports the scope it achieved, not the one it was asked for. A run that fell back reports scope: :all with the request kept in requested_scope, so a caller that refuses to move a baseline on a scoped run does not have to reason about fallbacks.

Uncommitted work counts

An agent mid-task has everything uncommitted, so a diff that reads only committed history reports no changes on exactly the runs this exists for. :changed reads the working tree against the merge base with the base ref, and folds in untracked files.

Example

ExQuality.Scope.resolve(:changed, base_ref: "origin/main")
#=> %{
#=>   scope: :changed,
#=>   requested_scope: :changed,
#=>   files: ["test/user_test.exs"],
#=>   base_ref: "origin/main",
#=>   fallback_reason: nil
#=> }

Summary

Types

resolved()

What a scope resolved to.

t()

A requested scope.

Functions

default_base_ref()

Returns the ref :changed is measured against when nothing names one.

describe(arg1)

Renders a scope for the report and for human output.

from_config(config)

Returns the scope a loaded config asks the test stage for.

parse(scope)

Parses a scope written by a human, in .quality.exs or on the command line.

resolve(scope, opts \\ [])

Resolves a scope to the test files to run.

test_files(changed)

Maps changed files to the test files that cover them.

Types

resolved()

@type resolved() :: %{
  scope: :all | :changed | {:glob, String.t()},
  requested_scope: t(),
  files: :all | [String.t()],
  base_ref: String.t() | nil,
  fallback_reason: String.t() | nil
}

What a scope resolved to.

files is :all for a full suite, which is not the same as []: an empty list would be a run of nothing.

t()

@type t() :: :all | :changed | {:glob, String.t()}

A requested scope.

Functions

default_base_ref()

@spec default_base_ref() :: String.t() | nil

Returns the ref :changed is measured against when nothing names one.

The repository's own origin/HEAD is preferred, because a repository whose trunk is neither main nor master has still recorded which one it is. Returns nil outside a git repository, which resolve/2 turns into a full suite rather than a failure.

describe(arg1)

@spec describe(t()) :: String.t()

Renders a scope for the report and for human output.

iex> ExQuality.Scope.describe(:all)
"all"

iex> ExQuality.Scope.describe({:glob, "test/unit/**"})
"test/unit/**"

from_config(config)

@spec from_config(keyword()) :: t()

Returns the scope a loaded config asks the test stage for.

Raises when the config names something that is not a scope, rather than falling back to :all: a typo that silently ran everything would be slow, and one that silently ran nothing would be a green run of no tests.

iex> ExQuality.Scope.from_config([])
:all

iex> ExQuality.Scope.from_config(test: [scope: :changed])
:changed

parse(scope)

@spec parse(term()) :: {:ok, t()} | {:error, String.t()}

Parses a scope written by a human, in .quality.exs or on the command line.

Anything that is not all or changed is a glob, because a glob is the only one of the three that has to carry a value.

iex> ExQuality.Scope.parse("changed")
{:ok, :changed}

iex> ExQuality.Scope.parse(:all)
{:ok, :all}

iex> ExQuality.Scope.parse("test/unit/**/*_test.exs")
{:ok, {:glob, "test/unit/**/*_test.exs"}}

iex> ExQuality.Scope.parse(42)
{:error, "test scope must be :all, :changed or a glob string, got: 42"}

resolve(scope, opts \\ [])

@spec resolve(
  t(),
  keyword()
) :: resolved()

Resolves a scope to the test files to run.

Options

:base_ref - what :changed is measured against (default: the repository's default branch, see default_base_ref/0)

test_files(changed)

@spec test_files([String.t()]) :: [String.t()]

Maps changed files to the test files that cover them.

A changed test file is itself. A changed source file is the test file beside it, lib/foo/bar.ex to test/foo/bar_test.exs, which works unchanged in an umbrella because apps/web/lib/foo.ex maps under apps/web/test/.

Only files that exist are returned. Everything else - a changed mix.exs, a source file with no test - contributes nothing, and contributing nothing is what makes resolve/2 fall back to the full suite.

iex> ExQuality.Scope.test_files(["mix.exs"])
[]