Testing a script in a repository of its own: an ExUnit setup callback and the functions a script test calls, over YmerNode.Script.Harness. Nothing is used — as with Req.Test, a test module names what it wants in setup.

The recipe

Take ymer_node as a dependency under runtime: false, and list :req among your own dependencies for runtime: a script's requests go through Req in the test VM, and without Req's application started a stubbed request exits with :noproc. One line of configuration stays yours, because no dependency can set it for the application that takes it:

config :elixir, :time_zone_database, Tz.TimeZoneDatabase

Without it every zone-aware DateTime call in a test answers {:error, :utc_only_time_zone_database}. Then, in each test module:

use ExUnit.Case, async: false

setup {YmerNode.Script.Test, :start_run_tree}

test "reads one issue" do
  compiled = YmerNode.Script.Test.compile_script("scripts/jira.exs")
  YmerNode.Script.Test.put_secrets(%{"JIRA_PASSWORD" => "probe-password"})

  Req.Test.stub(YmerNode.Script.Context, fn conn ->
    Req.Test.json(conn, %{"key" => "X-1"})
  end)

  context = YmerNode.Script.Test.context(secrets: ["JIRA_PASSWORD"])

  assert {:ok, _issue} = compiled.module.run(:get, %{"key" => "X-1"}, context)
end

async: false, and why

Every test module using start_run_tree/1 runs async: false. The run tree's names (YmerNode.Script.Harness.run_tree/0) are the node's own and VM-global, and so are the secrets file and the files directory: a second module overlapping a first would have start_run_tree/1 raise — ExUnit.Callbacks.start_supervised!/1 fails loudly on the name collision rather than returning a value — and would read a secrets file the first one's exit had just removed. Nothing enforces the flag — each test module writes it.

What the setup fills in

A key the application's configuration sets is never replaced. For a key it leaves unset — missing, or set to nil, and a plug set to false, which Req reads as no plug — start_run_tree/1 puts one through the harness for the test, and puts the configuration back as it found it when the test ends:

• the request plug — {Req.Test, YmerNode.Script.Context}, the name a test stubs under, so a test that forgets to stub meets Req.Test's own failure rather than the real host;
• the secrets file — tmp/secrets.env under the directory the test run started in, removed at setup — put_secrets/1 removes it again on exit — so nothing an interrupted run wrote there reaches the next test;
• the files directory — tmp/files there, which the setup creates whether it filled the key or not, as the node creates its own at boot.

Under mix test --partitions, each instance's MIX_TEST_PARTITION joins both names — tmp/secrets2.env and tmp/files2 for the second — so no two instances share a file.