Deterministic, scripted adapter for content-moderation testing. Implements
ALLM.ModerationAdapter.
Layer B — runtime. FakeModeration is the canonical testing moderation
adapter; it ships in lib/ (not test/support/) because users need it for
their own application tests, mirroring the ALLM.Providers.Fake,
ALLM.Providers.FakeImages, and ALLM.Providers.FakeEmbeddings precedent.
What FakeModeration is (and isn't)
FakeModeration mostly ignores the %ALLM.ModerationRequest{} passed to
moderate/2. It inspects :input to enforce the empty-input and
batch-size gates the ALLM.ModerationAdapter contract requires of every
implementation, and to size the default verdict; it reads :model and
:metadata to round-trip them onto the response. Otherwise the scripted
response is produced irrespective of the request — in particular no
scripted verdict is derived from the input text, and a {:ok, results}
entry decides how many results come back, not length(input).
Default verdict (no script)
With no script — adapter_opts[:moderation_script] absent or [] —
every input yields one unflagged ALLM.ModerationResult carrying all 13
omni-moderation category names, every value false and every score
0.0. Cardinality follows ALLM.ModerationRequest's rule: length(input)
results for an all-strings input, exactly one result for a multimodal one.
This is a deliberate divergence from ALLM.Providers.FakeEmbeddings, whose
no-script call surfaces an error: a clean verdict is a meaningful default
that costs a caller nothing, whereas a synthesized embedding vector is
not.
Spent script
A non-empty script whose cursor has run off the end is a different case and does NOT fall back to the clean verdict — it returns
{:error, %ALLM.Error.ModerationAdapterError{
reason: :unknown,
metadata: %{cause: :moderation_script_exhausted}}}"I didn't script anything, give me a benign default" is a convenience;
"my script ran out" is almost always an off-by-one in the caller's
expectation of how many times moderate/2 gets invoked, and a test vehicle
that answers it with an unflagged pass hides that bug. In particular a
truncated [{:retry_until_call, 1}] script would otherwise report success
on call 1 with no retry ever exercised.
Script shapes
opts[:adapter_opts][:moderation_script] accepts a list of script entries.
See script/1 for the full grammar.
adapter_opts: [
moderation_script: [
{:ok, [%ALLM.ModerationResult{...}]},
{:flagged, ["violence"]},
{:error, %ALLM.Error.ModerationAdapterError{reason: :rate_limited}},
{:retry_until_call, 3}
]
]{:flagged, categories} is the shorthand for the overwhelmingly common
test — "assert my app rejects flagged content" — and synthesizes a single
flagged result with those category names true at score 1.0 and every
other omni category false at 0.0.
{:retry_until_call, n} returns a synthetic
%ALLM.Error.ModerationAdapterError{reason: :rate_limited, retry_after_ms: 0}
for the first n - 1 calls against this entry, then advances the cursor to
the next entry on call n. Vehicle for testing ALLM.Retry.run/3
integration. Consecutive {:retry_until_call, _} entries chain: the
call that exhausts one entry's budget lands on the next entry and opens its
budget, which is how a layered retry budget is scripted.
Cursor behaviour
Multi-call scripts (:moderation_script) advance a per-process cursor on
every call. The cursor lives in the process dictionary at
{:allm_fake_moderation_cursor, key_id}, isolated per ExUnit test process
(async: true), GC'd on pid-down, zero-setup for the common case. The
key_id is chosen by this precedence:
adapter_opts[:script_cursor]— an explicit Agent pid (handled separately; seestart_script_cursor/0).adapter_opts[:cursor_key]— the engine's stable:id, injected by the façade dispatch chokepoint viaALLM.Engine.put_cursor_key/2.:erlang.phash2(script)— the content-hash fallback for direct adapter calls with no engine.
At the façade the cursor keys on engine identity, so two engines built with
content-equal :moderation_script values each read index 0 on their first
call, even in the same process. The content-hash footgun remains only for
DIRECT adapter calls — ALLM.Providers.FakeModeration.moderate(req, opts)
invoked without an engine receives no :cursor_key. Workaround for that
path: pass distinct adapter_opts[:script_cursor] Agent pids from
start_script_cursor/0.
Test-only capture seam
Pass adapter_opts[:capture_pid] with a pid to receive a side-channel
message every time moderate/2 is invoked, BEFORE any gate runs and before
the script is consulted. The message has the form:
{ALLM.Providers.FakeModeration, :call, %{request: request, opts: opts}}This is purely a side-channel — it does NOT affect the response. It exists
so test files can assert on what the adapter received without the
Process.register/2 + named-pid pattern (which forces async: false).
Examples
iex> req = ALLM.ModerationRequest.new(input: ["is this ok?"])
iex> opts = [adapter_opts: [moderation_script: [{:flagged, ["violence"]}]]]
iex> {:ok, resp} = ALLM.Providers.FakeModeration.moderate(req, opts)
iex> ALLM.ModerationResponse.flagged_categories(resp)
["violence"]
Summary
Types
One scripted moderation result.
Functions
The 13 omni-moderation category names FakeModeration reports, sorted.
Read the current cursor index for an Agent-backed cursor. Used in tests to assert how many calls have been consumed.
Return the maximum number of inputs FakeModeration accepts per call.
Execute a scripted moderation request.
Document and validate the script grammar for
adapter_opts[:moderation_script].
Start an Agent-backed script cursor for cross-process multi-call scripting and for disambiguating content-equal scripts in the same process.
Types
@type script_entry() :: {:ok, [ALLM.ModerationResult.t()]} | {:flagged, [String.t()]} | {:error, ALLM.Error.ModerationAdapterError.t()} | {:retry_until_call, pos_integer()}
One scripted moderation result.
Functions
@spec categories() :: [String.t()]
The 13 omni-moderation category names FakeModeration reports, sorted.
Exposed so tests can assert against the same vocabulary the adapter synthesizes rather than hand-copying a 13-element list.
Examples
iex> ALLM.Providers.FakeModeration.categories |> length()
13
iex> "self-harm/intent" in ALLM.Providers.FakeModeration.categories
true
@spec cursor_index(pid()) :: non_neg_integer()
Read the current cursor index for an Agent-backed cursor. Used in tests to assert how many calls have been consumed.
Examples
iex> pid = ALLM.Providers.FakeModeration.start_script_cursor
iex> req = ALLM.ModerationRequest.new(input: ["x"])
iex> opts = [adapter_opts: [moderation_script: [{:flagged, ["hate"]}], script_cursor: pid]]
iex> {:ok, _} = ALLM.Providers.FakeModeration.moderate(req, opts)
iex> ALLM.Providers.FakeModeration.cursor_index(pid)
1
@spec max_batch_size() :: pos_integer()
Return the maximum number of inputs FakeModeration accepts per call.
Deliberately not a provider-shaped number. The real moderation cap is
undocumented, and a mirrored constant would force every conformance run to
build a list of that size just to cross the :batch_too_large boundary.
32 is large enough to be plausible and small enough to be cheap.
Examples
iex> ALLM.Providers.FakeModeration.max_batch_size
32
@spec moderate( ALLM.ModerationRequest.t(), keyword() ) :: {:ok, ALLM.ModerationResponse.t()} | {:error, ALLM.Error.ModerationAdapterError.t()}
Execute a scripted moderation request.
Gate order, all before the script is consulted:
adapter_opts[:capture_pid]side-channel (fires even for rejected calls).input: []→{:error, %ModerationAdapterError{reason: :invalid_request}}.- item count
> max_batch_size()→{:error, %ModerationAdapterError{reason: :batch_too_large, metadata: %{count: n, max: max}}}. Per contract invariant 5 the count is the item count invariant 3 defines —length(input)for an all-strings input,1for a multimodal one — not the raw list length.
Otherwise reads the script from opts[:adapter_opts][:moderation_script],
advances the process-local cursor, and interprets the entry. An absent
script produces the default clean verdict; a spent non-empty script
returns :moderation_script_exhausted. Both are described in the module
docs.
Propagates opts[:request_id] onto response.request_id, request.model
onto response.model, request.metadata onto response.metadata, and
stamps provider: :fake.
Examples
iex> req = ALLM.ModerationRequest.new(input: ["hello"], metadata: %{trace: "t1"})
iex> {:ok, resp} = ALLM.Providers.FakeModeration.moderate(req, request_id: "rid-1")
iex> {resp.request_id, resp.metadata, ALLM.ModerationResponse.flagged?(resp)}
{"rid-1", %{trace: "t1"}, false}
iex> req = ALLM.ModerationRequest.new(input: [])
iex> {:error, err} = ALLM.Providers.FakeModeration.moderate(req, [])
iex> err.reason
:invalid_request
@spec script([script_entry()]) :: :ok
Document and validate the script grammar for
adapter_opts[:moderation_script].
Each entry is one of:
{:ok, [%ALLM.ModerationResult{}, ...]}— return the listed results verbatim.{:flagged, ["violence", ...]}— synthesize ONE flagged result with those category namestrueat score1.0, every other omni categoryfalseat0.0.{:error, %ALLM.Error.ModerationAdapterError{}}— return the struct verbatim.{:retry_until_call, n}— synthetic:rate_limitedfor the firstn - 1calls against this entry. Consecutive entries of this shape chain into a layered budget (the call that exhausts one entry opens the next entry's budget).
Returns :ok when the script is well-formed; raises ArgumentError on the
first invalid entry. (Validation is opt-in — the runtime moderate/2 path
tolerates a mix of legal entries and surfaces
metadata: %{cause: :moderation_script_exhausted} when the cursor runs off
the end of a non-empty script.)
Examples
iex> ALLM.Providers.FakeModeration.script([{:flagged, ["violence"]}])
:ok
@spec start_script_cursor() :: pid()
Start an Agent-backed script cursor for cross-process multi-call scripting and for disambiguating content-equal scripts in the same process.
Pass the returned pid as adapter_opts[:script_cursor]; subsequent calls
increment the cursor on the Agent rather than on the process dictionary.
Examples
iex> pid = ALLM.Providers.FakeModeration.start_script_cursor
iex> ALLM.Providers.FakeModeration.cursor_index(pid)
0