LivebookTest (livebook_test v0.1.0)

Test your Livebook notebooks - "mix test for Livebooks".

LivebookTest discovers .livemd notebooks, converts them to executable Elixir scripts, runs them, and reports failures. It supports local dependency overrides so standalone notebooks can be tested against the current repository checkout.

Quick start

Add to your Mix project:

def deps do
  [{:livebook_test, "~> 0.1.0", only: [:dev, :test], runtime: false}]
end

Run:

mix livebook.test

Pipeline

The library processes notebooks through a pipeline:

1. **Discovery** - find `.livemd` files via glob patterns
2. **Export** - convert notebooks to `.exs` scripts
3. **Patch** - optionally rewrite Mix.install deps to local paths
4. **Run** - execute each script as a subprocess
5. **Report** - summarize results and return CI exit codes

Local dependency testing

The key feature: notebooks that use Mix.install can be automatically patched to use your local checkout instead of published Hex packages.

# In config/config.exs
config :livebook_test,
  dependency_mode: :local,
  local_deps: [
    my_lib: "."
  ]

Programmatic API

LivebookTest.run()
LivebookTest.run(paths: ["examples/**/*.livemd"], mode: :local, timeout: 120_000)

Configuration

See LivebookTest.Config for full configuration options.

Summary

Types

run_option()

Options accepted by run/1

run_result()

Result of a complete test run

Functions

run(opts \\ [])

Discovers, exports, patches, runs, and reports on Livebook notebooks.

run_and_report(opts \\ [])

Convenience function that runs the pipeline and prints the report.

run_with_config(config, opts \\ [])

Runs the full pipeline using a pre-resolved config.

Types

run_option()

@type run_option() ::
  {:paths, [String.t()]}
  | {:exclude, [String.t()]}
  | {:mode, LivebookTest.Config.dependency_mode()}
  | {:timeout, non_neg_integer()}
  | {:local_deps, LivebookTest.Config.local_deps()}
  | {:verbose, boolean()}

Options accepted by run/1

run_result()

@type run_result() :: {LivebookTest.Config.t(), LivebookTest.Report.t()}

Result of a complete test run

Functions

run(opts \\ [])

@spec run([run_option()]) :: run_result()

Discovers, exports, patches, runs, and reports on Livebook notebooks.

This is the primary entry point. It orchestrates the full pipeline and returns a tuple with the resolved config and the report.

Options

:paths - list of glob patterns (default from config)
:exclude - list of glob patterns to exclude from discovery
:mode - :remote or :local dependency mode
:timeout - per-notebook timeout in milliseconds
:local_deps - dependency name → path mapping
:verbose - enable verbose output

Examples

iex> {config, report} = LivebookTest.run(paths: ["examples/**/*.livemd"])
iex> is_struct(config, LivebookTest.Config) and is_struct(report, LivebookTest.Report)
true

run_and_report(opts \\ [])

@spec run_and_report([run_option()]) :: 0 | 1 | 2

Convenience function that runs the pipeline and prints the report.

Returns the exit code (0 for success, 1 for failure) suitable for CI/CD use.

Examples

iex> LivebookTest.run_and_report(paths: ["examples/**/*.livemd"]) in [0, 1, 2]
true

run_with_config(config, opts \\ [])

@spec run_with_config(
  LivebookTest.Config.t(),
  keyword()
) :: LivebookTest.Report.t()

Runs the full pipeline using a pre-resolved config.

Useful when you need fine-grained control over configuration before running.

Examples

iex> config = LivebookTest.Config.resolve(paths: ["examples/**/*.livemd"])
iex> report = LivebookTest.run_with_config(config)
iex> is_struct(report, LivebookTest.Report)
true