Cite (Cite v0.2.0)

Finds what a policy describes in a document and cites the caller's own passages behind each finding, unchanged.

Three steps:

defmodule MyApp.Riddles do                       # declare the policy once
  use Cite.Policy

  concern :riddle do
    detect do
      question "Does {passage} pose a riddle?"
      yes "A question asked to be puzzled over."
      no "A plain question, a statement, or a remark."
    end
  end
end

source = Cite.source(lines)                      # hand over the passages
report = Cite.judge(client, source, MyApp.Riddles) # judge the source against it

judge/4 runs two fixed rounds: a screen of every passage for every detect, then one judgment per finding. A decision model answers typed questions about the passages. It never writes text and never sees a passage the caller did not list.

The client

client/2 builds one from a Cite.Provider module. Any 1-arity function of the same shape works in its place:

request -> {:ok, verdict} | {:error, reason}

request is %{"state" => map(), "questions" => map()}. A value in "state" may be a Cite.Wire.Object, a JSON object that keeps its keys in order. verdict is %{answers: map(), usage: usage | nil}, plus :model, the versioned id of the model that answered, when the provider reports it.

judge/4 calls the client from task processes, up to concurrency at once (4 by default), so the client must be safe to call concurrently. Each task starts with the caller's Logger metadata and process level. Cite carries nothing else over: OpenTelemetry context, for one, doesn't reach the task, so a client that needs it attaches it itself. In the client, self() is the task, not the caller.

Tasks don't trap exits. A crash in a process the client links to ends that request's task with the crash's reason. A caller that doesn't trap exits dies of it at once, through its link to the task. In a caller that traps exits, such as a GenServer, judge/4 exits with the same reason, an exit the caller can catch. As with a raise, it comes once the requests before it have answered, and stops the rest as a raise does.

Summary

Types

client()

usage()

verdict()

Functions

client(provider, opts \\ [])

A client backed by provider, a module implementing Cite.Provider. opts are the provider's options. Build it once, where the credentials live, and pass it in.

judge(client, source, policy_module, opts \\ [])

Judges source against policy_module, a module that uses Cite.Policy.

source(units, opts \\ [])

The source: the caller's passages, each a text or %{text: text} with optional :id and :meta. See Cite.Source.

Types

client()

@type client() :: (map() -> {:ok, verdict()} | {:error, term()})

usage()

@type usage() :: Cite.Report.usage()

verdict()

@type verdict() :: %{
  :answers => map(),
  :usage => usage() | nil,
  optional(:model) => String.t()
}

Functions

client(provider, opts \\ [])

@spec client(module(), keyword()) :: client()

A client backed by provider, a module implementing Cite.Provider. opts are the provider's options. Build it once, where the credentials live, and pass it in.

client = Cite.client(Cite.Provider.TypeSafe, api_key: System.fetch_env!("TYPESAFE_API_KEY"))

Raises ArgumentError if provider does not implement Cite.Provider or opts is not a keyword list. The provider's new/1 raises on options it refuses.

judge(client, source, policy_module, opts \\ [])

@spec judge(client(), Cite.Source.t(), module(), keyword()) :: Cite.Report.t()

Judges source against policy_module, a module that uses Cite.Policy.

Before any request, raises ArgumentError on an argument it cannot use: a client that is not a 1-arity function, a source not built by source/2, a module that is not a policy, or a bad option.

Mid-run, raises RuntimeError when the client returns something outside its contract. judge/4 passes on, as is, whatever the client raises, throws, or exits with, an ArgumentError included. Either comes once the requests before it have answered. Once a call raises, throws, or exits, no new request starts. A request before it still finishes, including the halves Cite sends when it is too large and any retries the client makes. One after it is stopped at once, but the provider may already have received, and billed, it.

Options

:threshold - A number from 0 to 1: the round-1 score a filter, factor, or detect must exceed. It decides which passages are gathered. A finding's confirms read every passage gathered with it, so the threshold changes verdicts as well as recall. Measured on transcripts, a threshold below 0.5 invents findings in concerns with no match at 0.5 and multiplies review load. The default value is 0.5.
:review_band - {low, high}, 0 <= low < high <= 1. At or below low, a passage is dropped and a check fails. At or above high, each holds. The default value is {0.4, 0.6}.
:window (pos_integer/0) - Passages per round-1 request, a positive integer. At 1, each request holds one passage. The default value is 40.
:concurrency (pos_integer/0) - The most requests in flight at once within a round. The rounds run one after the other. Given the same replies, the report is the same at any setting. Set it within the provider's rate limits. At 1, requests go out one at a time. The default value is 4.

source(units, opts \\ [])

@spec source([String.t() | map()], keyword()) :: Cite.Source.t()

The source: the caller's passages, each a text or %{text: text} with optional :id and :meta. See Cite.Source.

Options

:as - The key the passages sit under in every request, such as "utterances". It is part of every path the model reads, so it must be UTF-8 with no whitespace, ., backtick, [, or ]. Passage ids follow the same rule. The default value is "passages".
:show - The meta keys the model sees beside id and text, as given in meta (:speaker finds %{speaker: _}, not %{"speaker" => _}). It may not name id or text, or one key as both an atom and a string. The default value is [].