Compliance suite for LemonGateway.Engine implementations.

What an engine is

An engine is the thing that actually answers. LemonGateway decides that a run should happen — concurrency, locking, scheduling, resume — and hands the work to an engine, which wraps a CLI tool (codex, claude, droid), an API, or an in-process agent. LemonGateway.Engines.Echo is the reference implementation and is 60 lines long; CodingAgent.GatewayEngine is a real one that lives in a different application entirely and registers itself at boot.

Engines do not return answers. They start work and stream events to a sink process, which is what makes a run observable while it is still running.

The contract

Identity

id/0 returns a slug matching ~r/^[a-z][a-z0-9_-]*$/ that is not "default" or "help" — LemonGateway.EngineRegistry reserves those two and refuses to start with an engine that claims either. The id appears in resume tokens, routing config and user-visible commands, so treat it as permanent.

Resume tokens round-trip

Three callbacks share one job: letting a later run continue an earlier session.

• format_resume/1 renders a LemonCore.ResumeToken as the literal command a user could paste ("codex resume thread_abc", "claude --resume xyz").
• extract_resume/1 finds your token in arbitrary text — typically the tail of your own tool's output — and returns a %LemonCore.ResumeToken{} or nil. It must return nil, never raise, for text that is not yours, and it must not claim another engine's tokens.
• is_resume_line/1 answers whether a line is only a resume command, so renderers can keep it when truncating output.

The suite checks the cycle: a token you formatted must be extractable, must come back with your id/0 and the same value, and must be recognised as a resume line.

Runs are asynchronous and event-driven

start_run/3 returns {:ok, run_ref, cancel_ctx} immediately — before the work finishes — or {:error, reason}. Everything the caller learns after that arrives as messages to sink_pid:

{:engine_event, run_ref, event}   # LemonGateway.Event.started/action_event/completed
{:engine_delta, run_ref, text}    # streaming text

run_ref ties messages to the run; cancel_ctx is opaque state you will be handed back. Emit exactly one started event and, eventually, exactly one completed event — the gateway's run lifecycle hangs on the completed event arriving even for failures (ok: false), so an engine that dies silently leaves a run wedged until the watchdog fires.

cancel/1 is total

The gateway calls cancel(cancel_ctx) and expects :ok — for the exact term your start_run/3 returned, for a stale or partial one it is holding after a crash or restart, and for a run that already finished. Cancelling twice is :ok too. Pattern-matching only your happy-path context turns a routine cancellation into a FunctionClauseError in the caller, so end your clauses with def cancel(_ctx), do: :ok.

Steering is optional but must be declared honestly

supports_steer?/0 is a required callback; steer/2 is optional. If you answer true you must export steer/2 and accept mid-run text, returning :ok or {:error, reason}.

Minimal implementation

defmodule MyApp.Engine do
  @behaviour LemonGateway.Engine

  alias LemonCore.ResumeToken
  alias LemonGateway.Event

  @impl true
  def id, do: "myengine"

  @impl true
  def format_resume(%ResumeToken{value: value}), do: "myengine resume #{value}"

  @impl true
  def extract_resume(text) do
    case Regex.run(~r/myengine\s+resume\s+([\w-]+)/i, text) do
      [_, value] -> %ResumeToken{engine: id(), value: value}
      _ -> nil
    end
  end

  @impl true
  def is_resume_line(line), do: Regex.match?(~r/^\s*myengine\s+resume\s+[\w-]+\s*$/i, line)

  @impl true
  def supports_steer?, do: false

  @impl true
  def start_run(job, _opts, sink_pid) do
    run_ref = make_ref()
    resume = job.resume || %ResumeToken{engine: id(), value: "session-1"}

    {:ok, pid} =
      Task.start(fn ->
        send(sink_pid, {:engine_event, run_ref, Event.started(%{engine: id(), resume: resume})})
        answer = MyApp.Api.complete(job.prompt)

        send(
          sink_pid,
          {:engine_event, run_ref,
           Event.completed(%{engine: id(), resume: resume, ok: true, answer: answer})}
        )
      end)

    {:ok, run_ref, %{task_pid: pid}}
  end

  @impl true
  def cancel(%{task_pid: pid}) when is_pid(pid) do
    Process.exit(pid, :kill)
    :ok
  end

  def cancel(_ctx), do: :ok
end

Registering from another application

An engine that ships outside lemon_gateway registers itself when its own application starts:

LemonGateway.EngineRegistry.register(MyApp.Engine)

Registration is idempotent, validates the id, and survives a registry restart. This suite performs that round-trip (see the :registry option).

Running the suite

defmodule MyApp.EngineComplianceTest do
  use LemonPlatformTest.EngineCase, async: false, engine: MyApp.Engine
end

By default this runs the static contract only: identity, resume-token cycle, callback consistency, registry round-trip. Nothing is executed. Pass a :run_probe to also exercise the run lifecycle, which is worth doing for any engine that can answer without a network call or a real model:

defmodule MyApp.EngineComplianceTest do
  use LemonPlatformTest.EngineCase,
    async: false,
    engine: MyApp.Engine,
    run_probe: {__MODULE__, :job}

  def job(_context) do
    %LemonGateway.Types.Job{run_id: "compliance", prompt: "ping", engine_id: "myengine"}
  end
end

Options

• :engine — required, the engine module under test.
• :resume_value — the token value used for the round-trip. Default "session-abc123". Must be something your extract_resume/1 accepts; keep it to [A-Za-z0-9_-].
• :registry — round-trip the engine through LemonGateway.EngineRegistry. Default true, because "works standalone, invisible to the platform" is the integration failure this suite exists to catch. It is also the only option that is on by default and touches global state, so it is worth knowing exactly what it does: it starts :lemon_gateway if nothing else has (with the health listener disabled, so no port is bound), and the gateway application's own startup in turn points LemonCore.EngineInfoBridge at the gateway's registries for the rest of the VM's life. The engine list it registers into is snapshotted and restored per test. Pass registry: false in a suite where starting the gateway is not acceptable.
• :run_probe — {Module, :function} returning a %LemonGateway.Types.Job{}, called with the test context. Enables the lifecycle tests: start_run/3 returns a ref, a started event arrives at the sink, and cancel/1 returns :ok. Only pass this if running the job is free and offline.
• :run_timeout — milliseconds to wait for the started event. Default 2_000.
• :cancel_tolerates_unknown_ctx — assert that cancel/1 returns :ok for a context it did not produce. Default true, because LemonGateway.Engine.cancel/1 requires it. Set it to false only to quarantine a known-strict engine you have not fixed yet; that is a departure from the contract, not a supported configuration.

Known gaps in the behaviour

• steer/2 is optional but supports_steer?/0 is not. An engine that answers false still has to define supports_steer?/0; nothing stops an engine from answering true and omitting steer/2 except this suite, because making steer/2 mandatory would break every engine that does not steer.
• start_run/3's opts are a loose map. run_opts() documents :cwd, :env, :timeout_ms and :capabilities as optional keys, and engines differ on which they honour. The suite passes %{} and does not assert how options are interpreted.