TypeSafe Jev for OTP.
Questions are data, the reply is a plain map, and Jev.Server lets a
GenServer talk to Jev by replying:
def handle_call({:labels, issue}, from, s) do
{:reply, {from, issue,
kind: {"What kind of issue?", %{bug: nil, feature: nil, other: nil}},
security: "Is this a vulnerability?"}, s}
end
def handle_answer(%{security: p}, from, s) when p > 0.5, do: done(from, [:security], s)
def handle_answer(%{kind: k, confidence: %{kind: c}}, from, s) when c > 0.85, do: done(from, [k], s)
def handle_answer(%{kind: k}, from, s), do: done(from, [k, :"needs-triage"], s)This module holds the pure half: questions/1 normalizes shorthands into
Jev.Noul, Jev.Choice, and Jev.Score structs, and reply/3 turns a
decoded API response into the reply map. Jev.Wire describes the response
body, and Jev.HTTP.post/3 is the transport.
The reply map
%{
kind: :bug, # Choice → label atom
severity: 2.4, # Score → expected level, float
security: 0.03, # Noul → probability of yes
confidence: %{kind: 0.91, severity: 0.62},
probabilities: %{kind: %{bug: 0.93, feature: 0.04, other: 0.03},
severity: %{0 => 0.1, 1 => 0.1, 2 => 0.2, 3 => 0.6}},
usage: %{input_tokens: 812, output_tokens: 0, cost: 3.4e-5},
model: "jev-1.13.0" # the concrete model that answered, even when you asked for jev-latest
}confidence, probabilities, usage, and model are reserved question names.
Other models
The wire format is served by open decision models as well as by TypeSafe,
and reply/3 accepts what they send: usage may be missing, unknown fields
are ignored, and when an answer carries probabilities but no confidence,
confidence is computed the way TypeSafe defines it, as the top probability
normalized over the number of options, (top - 1/k) / (1 - 1/k). A body
that does not fit Jev.Wire at all is a JSONCodec.Error naming the field.
Servers are calibrated differently, so a threshold tuned on one model is a
starting point on another, not a guarantee.
Summary
Functions
TypeSafe's confidence for a distribution over options options.
The cost in USD of input_tokens at usd_per_million_input.
Normalizes a keyword list or map of questions into question structs.
Turns a response body into the reply map.
Types
Text, a JSON-encodable map or list, or nil.
@type question() :: Jev.Noul.t() | Jev.Choice.t() | Jev.Score.t()
@type shorthand() :: entry() | {entry(), %{required(atom()) => entry()}} | {entry(), [entry(), ...]} | question()
A question in one of the accepted shorthands:
- a string or map:
Jev.Noulinstructions {instructions, %{label => description}}:Jev.Choice{instructions, [level, ...]}:Jev.Score- a question struct, passed through
@type usage() :: %{ input_tokens: non_neg_integer(), output_tokens: non_neg_integer(), cost: float() }
Functions
@spec confidence(%{required(term()) => number()}, pos_integer() | nil) :: float()
TypeSafe's confidence for a distribution over options options.
The top probability, rescaled so that a uniform distribution is 0 and
certainty is 1: (top - 1/k) / (1 - 1/k). options defaults to the size of
the distribution; pass the number of criteria when a server omits options
with zero probability.
iex> Jev.confidence(%{yes: 0.75, no: 0.25})
0.5
@spec cost(non_neg_integer(), number()) :: float()
The cost in USD of input_tokens at usd_per_million_input.
Jev bills input tokens only. The price defaults to TypeSafe's, 0.042 USD per
million, and can be set with config :jev, usd_per_million_input: 0.042.
Named endpoints carry their own price, zero unless configured.
iex> Jev.cost(500_000, 1.0)
0.5
Normalizes a keyword list or map of questions into question structs.
iex> Jev.questions(security: "Is this a vulnerability?")
%{security: %Jev.Noul{instructions: "Is this a vulnerability?"}}
iex> Jev.questions(kind: {"What kind?", %{bug: "Broken", other: nil}})
%{kind: %Jev.Choice{instructions: "What kind?", criteria: %{bug: "Broken", other: nil}}}
iex> Jev.questions(severity: {"How severe?", ["Cosmetic", "Blocks"]})
%{severity: %Jev.Score{instructions: "How severe?", criteria: ["Cosmetic", "Blocks"]}}Raises ArgumentError for an empty list, a reserved name, a choice outside
2..255 options, or a score outside 2..10 levels.
@spec reply(map() | Jev.Wire.Response.t(), questions(), [ {:usd_per_million_input, number()} ]) :: reply()
Turns a response body into the reply map.
The body is a decoded JSON map or an already decoded Jev.Wire.Response.
A map that does not fit the wire format raises JSONCodec.Error.
The questions are needed to map labels back to atoms: the criteria keys are
the only atoms this function can produce, so no atoms are created from input.
An answer whose kind does not match its question raises ArgumentError.
usd_per_million_input: prices the usage; it defaults to the configured
price, see cost/2.