ALLM.Providers.TypeSafe.Classification (allm v0.6.1)

Copy Markdown View Source

TypeSafe typed-classification adapter (Jev). Implements ALLM.ClassificationAdapter.

Layer B — runtime. Jev is a classification model, not a chat model: it answers closed-form questions about a piece of state with calibrated probabilities and generates no text. This adapter is therefore a classification adapter only; TypeSafe has no chat, image, embeddings, moderation or audio adapter in ALLM.

engine =
  ALLM.Engine.new(
    classification_adapter: ALLM.Providers.TypeSafe.Classification,
    classification_model: "jev-1.13.0"
  )

{:ok, resp} = ALLM.classify(engine, ticket_text, questions: questions)

Keys resolve through ALLM.Keys.fetch!(:typesafe, opts) at request-build time, from opts[:api_key] or the TYPESAFE_API_KEY environment variable. No key ever lives on the engine.

:yes_no is TypeSafe's noul

TypeSafe calls its yes/no question type noul. ALLM's provider-neutral name for it is :yes_no: this adapter sends "type": "noul" for a :yes_no question and decodes a noul answer into %ALLM.ClassificationAnswer{type: :yes_no, yes_probability: p}. The alias appears nowhere else in ALLM. A noul answer carries no confidence, so :confidence is nil on every :yes_no answer.

Wire-field map

ConcernTypeSafe
EndpointPOST https://api.typesafe.ai/v1/systemone (not overridable)
Authauthorization: Bearer <key>
Request{"state", "model", "questions": {"<id>": {"type", "instructions", "criteria"}}}
Question type:choice → "choice", :score → "score", :yes_no → "noul"
Choice criteria{"<option>": description | null}
Score criteriaan ordered array of levels, low to high
Yes/no criteriaoptional {"true": …, "false": …}; omitted when nil
Choice answerchoice, probabilities (by option), confidence
Score answerscore, probabilities and legend keyed "0".."n-1" (decoded to lists, index = level), confidence
Noul answernoul (the probability of yes)
Response modelmodel — the versioned id that answered ("jev-1.13.0"), not the alias sent
Usageusage.input_tokens, usage.output_tokens
Provider request idx-typesafe-request-id response header → ClassificationResponse.id
Errors{"detail": …}, where detail is a string, an object with message and error_type, or (on 422) a list of {"loc", "msg"} entries

A score question whose levels are objects gets those objects back as its :legend entries; a string level comes back as a string.

TypeSafe ignores unknown fields in a request, both on a question and at the top level, so a misspelled field is silently dropped rather than rejected.

Adapter-injected defaults

The wire requires model, and ALLM.ClassificationRequest allows model: nil. When the request carries no model this adapter sends "jev-latest". jev-latest is an alias that moves when TypeSafe ships a new version; pin a versioned id such as "jev-1.13.0" (on the engine's :classification_model, or on the request) when you tune thresholds against its probabilities. ALLM.classify/3 stamps engine.classification_model onto a request with no model before the adapter sees it, so the default applies only when neither is set.

request.options is ignored: nothing in it reaches the wire. request.metadata is returned unchanged on the response.

Limits and pre-flight gates

Before any HTTP I/O, and before ALLM.Keys.fetch!/2 (so a request that is going to be rejected never needs a key), in this order:

  1. Empty questions. questions: %{} → :invalid_request with metadata: %{field: :questions}.
  2. Provider limits. A choice question with more than 255 options, or a score question with more than 10 levels → :invalid_request with metadata: %{question: id, limit: n}. Both limits are TypeSafe's own; the provider answers 400 above them.
  3. List state. A list state must hold only strings (TypeSafe documents list state as "an array of text values") → :invalid_request with metadata: %{field: :state}.
  4. Encodability. A body that cannot be JSON-encoded (a tuple, a pid, an improper list anywhere in the state or a question) → :invalid_request with metadata: %{cause: :unencodable_body}. ALLM.classify/3 validates the same thing earlier; this gate is what a direct classify/2 call meets.

TypeSafe documents no maximum question count: a request with 512 questions was accepted when this adapter was recorded. Each question re-reads the state, so the per-request token budget (64k tokens in all; 32k for the state plus the longest question) is the practical limit. A request over it is :context_length_exceeded.

Errors

StatusReason
400 with detail.error_type == "max_tokens_exceeded":context_length_exceeded
400, 404, 422:invalid_request (TypeSafe answers 400 for an unknown question type, an unknown model and a limit breach, and 422 for a body that fails schema validation)
401, 403:authentication_failed
429:rate_limited, with retry_after_ms from Retry-After when present
500, 502, 503, 504, 529:provider_unavailable (529 is TypeSafe's "Overloaded")
any other:unknown

A transport timeout is :timeout; any other transport failure is :network_error. A 200 whose body does not match the questions asked (an answer missing, an extra answer, a type that differs from the question's, a choice outside the options, a score with the wrong number of levels or outside its levels, a probability or confidence outside 0..1) is :malformed_response.

Error :metadata carries status, typesafe_error_type (the body's detail.error_type, when present) and typesafe_request_id (the x-typesafe-request-id header, which TypeSafe support asks for).

Error-struct hygiene

%ALLM.Error.ClassificationAdapterError{} is JSON-encodable and is commonly logged and persisted. This adapter never stores an exception, a response body or a request header in :cause or :metadata: every error it returns has cause: nil and encodes with Jason.encode!/1. A transport failure's reason atom goes in metadata.transport_reason. Every provider-authored string (the message, the error type, the request id) passes a redactor that removes the resolved API key literally and any apikey_…-shaped token by pattern. TypeSafe's 401 text does not echo the key; the redactor is defence in depth.

Retry integration

One HTTP attempt per call, with no retry loop here: ALLM.classify/3 retries :rate_limited, :provider_unavailable, :timeout and :network_error according to engine.retry. A direct classify/2 call makes exactly one attempt; a caller retrying it themselves can read retry_after_ms.

Test-injection escape hatch

classify/2 honours opts[:adapter_opts][:classification_script]: when that key holds any non-nil value (including []), the call delegates to ALLM.Providers.FakeClassification.classify/2 before any gate runs. This is what lets the ALLM.ClassificationAdapter conformance suite drive this adapter without an HTTP stub. The switch is keyed on that per-call option only. prepare_request/2 does not delegate; under a script it returns an error, since a scripted answer has no Req.Request analogue.

Summary

Functions

Classify request.state against every question in one call to TypeSafe.

Return an unfired Req.Request configured exactly as classify/2 would fire it, for callers who need to add headers or middleware before dispatch.

Functions

classify(request, opts)

Classify request.state against every question in one call to TypeSafe.

Returns {:ok, %ALLM.ClassificationResponse{}} with one answer per question id, or {:error, %ALLM.Error.ClassificationAdapterError{}}. Every HTTP-shaped failure converts, including transport errors. The one exception is ALLM.Keys.fetch!/2, which raises %ALLM.Error.EngineError{reason: :missing_key} when no key is found; all pre-flight gates run before it.

When request.model is nil the adapter sends "jev-latest". Pin a versioned id to keep probabilities stable across TypeSafe releases.

A body that cannot be JSON-encoded returns :invalid_request with metadata: %{cause: :unencodable_body} and makes no call.

See the module documentation for the gate order, the wire-field map, the noul alias and the error mapping.

Examples

iex> q = ALLM.ClassificationQuestion.yes_no("Is a refund requested?")
iex> req = ALLM.ClassificationRequest.new(state: "Refund me.", questions: %{"refund" => q})
iex> opts = [adapter_opts: [classification_script: [{:answers, %{"refund" => 0.9}}]]]
iex> {:ok, resp} = ALLM.Providers.TypeSafe.Classification.classify(req, opts)
iex> ALLM.ClassificationResponse.answer(resp, "refund").yes_probability
0.9

iex> req = ALLM.ClassificationRequest.new(state: "hi", questions: %{})
iex> {:error, err} = ALLM.Providers.TypeSafe.Classification.classify(req, [])
iex> err.reason
:invalid_request

prepare_request(request, opts)

@spec prepare_request(
  ALLM.ClassificationRequest.t(),
  keyword()
) ::
  {:ok, Req.Request.t()} | {:error, ALLM.Error.ClassificationAdapterError.t()}

Return an unfired Req.Request configured exactly as classify/2 would fire it, for callers who need to add headers or middleware before dispatch.

The pre-flight gates run first, so this returns an error for a request classify/2 would reject before I/O. Under opts[:adapter_opts][:classification_script] it returns an :unknown error with metadata: %{cause: :scripted_adapter}.

Examples

iex> q = ALLM.ClassificationQuestion.yes_no("Is a refund requested?")
iex> req = ALLM.ClassificationRequest.new(state: "Refund me.", questions: %{"refund" => q})
iex> {:ok, http} = ALLM.Providers.TypeSafe.Classification.prepare_request(req, api_key: "apikey_x")
iex> URI.to_string(http.url)
"https://api.typesafe.ai/v1/systemone"