TypeSafe.Test (TypeSafe AI v0.1.0-alpha.1)

Copy Markdown View Source

Stub the TypeSafe API in your tests without fixtures or a network.

Add plug to your test dependencies ({:plug, "~> 1.16", only: :test}), then build a test client and describe the answers you want by question id:

# test/test_helper.exs
ExUnit.start()

# in a test
setup :typesafe_stubs

test "routes billing tickets" do
  client = TypeSafe.Test.client()

  TypeSafe.Test.stub(client,
    dept: {:choice, :billing, 0.9},
    urgent: {:noul, 0.3},
    anger: {:score, 1, 0.8}
  )

  assert {:ok, result} = MyApp.Triage.run(client, "Where is my refund?")
  assert result.answers.dept.choice == :billing
end

The stub reads the questions in each request and builds a response the way the API would, so the decoded structs are identical to those from a real call: probabilities sum to one, Score answers carry level, label and levels, and keys come back as atoms or strings exactly as you sent them.

Answer specs

  • {:noul, probability}
  • {:choice, option, confidence} - option is one of the question's keys; the remaining probability is spread evenly across the other options
  • {:score, level, confidence} - level is a 0-based index; the remaining probability is spread evenly across the other levels

A request that asks a question you did not stub raises, so a test cannot silently pass on a default answer.

Errors and models

TypeSafe.Test.stub_error(client, 429, %{"error" => "slow down"}, [{"retry-after", "1"}])
TypeSafe.Test.stub_models(client, [%{name: "jev-1.13.0", description: "...", release_date: "2026-01-01"}])

Concurrency

Stubs use Req.Test, which follows the ownership model of Mox: call Req.Test.set_req_test_from_context/1 in setup (or the typesafe_stubs/1 helper here) and stubs are private to each async test.

Summary

Types

How to answer one question.

Functions

Builds a client whose requests are served by this module's stubs.

Sends a JSON response from inside a custom Req.Test stub, for cases the built-in stubs do not cover

Stubs the evaluation endpoint with one answer per question id.

Stubs every request with an HTTP error response.

Stubs the models endpoint. Each entry needs a :name; :description and :release_date are optional.

An ExUnit setup callback: setup :typesafe_stubs.

Types

answer_spec()

@type answer_spec() ::
  {:noul, number()}
  | {:choice, atom() | String.t(), number()}
  | {:score, non_neg_integer(), number()}

How to answer one question.

Functions

client(opts \\ [])

@spec client(keyword()) :: TypeSafe.Client.t()

Builds a client whose requests are served by this module's stubs.

Options are passed to TypeSafe.new/1; api_key defaults to "test-key" and retries are disabled unless you set :retry. Pass :name to use a custom Req.Test stub name.

json(conn, status, body, headers \\ [])

@spec json(Plug.Conn.t(), pos_integer(), term(), [{String.t(), String.t()}]) ::
  Plug.Conn.t()

Sends a JSON response from inside a custom Req.Test stub, for cases the built-in stubs do not cover:

Req.Test.stub(TypeSafe.Test, fn conn ->
  TypeSafe.Test.json(conn, 200, %{"models" => []})
end)

stub(client, answers)

@spec stub(
  TypeSafe.Client.t(),
  keyword() | %{required(atom() | String.t()) => answer_spec()}
) ::
  TypeSafe.Client.t()

Stubs the evaluation endpoint with one answer per question id.

Returns the client so the call can be piped.

stub_error(client, status, body \\ %{}, headers \\ [])

@spec stub_error(TypeSafe.Client.t(), pos_integer(), term(), [
  {String.t(), String.t()}
]) ::
  TypeSafe.Client.t()

Stubs every request with an HTTP error response.

stub_models(client, models)

@spec stub_models(TypeSafe.Client.t(), [map()]) :: TypeSafe.Client.t()

Stubs the models endpoint. Each entry needs a :name; :description and :release_date are optional.

typesafe_stubs(context)

@spec typesafe_stubs(map()) :: :ok

An ExUnit setup callback: setup :typesafe_stubs.

Makes stubs private to async tests and shared otherwise.