ALLM.Providers.OpenAI.Embeddings (allm v0.5.0)

Copy Markdown View Source

OpenAI text-embeddings provider adapter — implements ALLM.EmbeddingAdapter against OpenAI's POST /v1/embeddings endpoint.

Layer B — runtime. Constructed via ALLM.Engine.new(embed_adapter: ALLM.Providers.OpenAI.Embeddings, model: "text-embedding-3-small") and consumed through ALLM.embed/3. Keys resolve via ALLM.Keys.fetch!(:openai, opts) at request-build time — no key ever lives on the engine.

req = ALLM.EmbeddingRequest.new(input: ["hello"], model: "text-embedding-3-small")
{:ok, resp} = ALLM.Providers.OpenAI.Embeddings.embed(req, api_key: "sk-...")
[[_ | _]] = ALLM.EmbeddingResponse.vectors(resp)

Wire-field map

ConcernOpenAI
EndpointPOST https://api.openai.com/v1/embeddings (not overridable)
Authauthorization: Bearer <key>
Inputinput — always sent as an array, even for one input
Modelmodel
Dimensionsdimensionstext-embedding-3 and later only
Task typenone:task_type is dropped (logged at :debug)
Truncatenone — see "Truncation" below
Vectorsdata[].embedding
Indexdata[].index
Usagetop-level usageinput_tokens ← prompt_tokens, total_tokens ← total_tokens, output_tokens = nil
Batch cap2048 array items (max_batch_size/0)

Adapter-injected defaults

None. The wire requires model, and ALLM.EmbeddingRequest permits model: nil. This adapter does not invent a default model: a nil :model is OMITTED from the body and OpenAI answers with a 400, surfaced as %ALLM.Error.EmbeddingAdapterError{reason: :invalid_request}. Guessing an embedding model would silently produce vectors of an unexpected dimensionality into a caller's vector column, which is unrecoverable after the fact. ALLM.embed/3 stamps the engine's resolved model onto the request before dispatch, so this only arises on a direct adapter call.

:task_type is dropped

ALLM.EmbeddingRequest's provider-neutral :task_type enum has no OpenAI equivalent. The adapter drops the field rather than erroring, and logs the drop at :debug. A caller who needs asymmetric query/document embedding should use a provider that supports it.

Truncation

:truncate is a no-op here, in both directions: OpenAI answers an over-length input with a 400 rather than silently truncating it, so there is no wire field to carry either value. The field is neither sent nor treated as an error.

Token budget vs. batch size

max_batch_size/0 is 2048 — the array-item cap. OpenAI separately caps a single request at 300,000 tokens summed across all inputs, which is reachable far below 2048 items when the inputs are long. That rejection arrives as a 400 and maps to %ALLM.Error.EmbeddingAdapterError{reason: :context_length_exceeded}. ALLM ships no tokenizer and will not guess token counts, so the recovery is yours: lower your effective batch size and re-drive ALLM.embed/3, or chunk against max_batch_size/0 yourself with a smaller stride.

Reduced dimensions

dimensions: is supported only on text-embedding-3 and later. Setting it on text-embedding-ada-002 is rejected pre-flight with %ALLM.Error.EmbeddingAdapterError{reason: :unsupported_feature, metadata: %{feature: :dimensions, model: model}} — the adapter can see the request is malformed without spending a round-trip.

Pre-flight gates

Before any HTTP I/O — and, deliberately, before ALLM.Keys.fetch!/2, so a request that is going to be rejected never needs a valid API key:

  1. Empty input. input: []:invalid_request.
  2. Batch size. length(input) > 2048:batch_too_large with metadata: %{count: n, max: 2048}. An :input that is not a list at all — OpenAI's own wire accepts a bare string, so it is the likeliest direct-adapter mistake — is rejected here as :invalid_request with metadata: %{field: :input} rather than raising.
  3. Feature support. dimensions: on a model with no such knob → :unsupported_feature.

Capability pre-flight against a model catalog is NOT performed here — it lives in ALLM.embed/3. A direct adapter call bypasses it by design.

Response ordering

data[] is sorted by :index before it becomes :embeddings. OpenAI documents that field precisely because array order is not contractual, and ALLM.EmbeddingResponse's order-correspondence invariant depends on it.

Request-id preservation

opts[:request_id] is reflected onto response.request_id unchanged. When it is absent, the adapter falls back to OpenAI's x-request-id response header. request.metadata round-trips onto response.metadata untouched.

The x-request-id fallback is unreachable through ALLM.embed/3

The façade always supplies opts[:request_id] (it generates one when the caller does not), so on that path the left branch always wins and OpenAI's own correlation id is never observed. It surfaces only on a direct embed/2 / decode_response/4 call that omits opts[:request_id]. Stamping it into response.metadata the way ALLM.Providers.OpenAI.Images does would contradict ALLM.EmbeddingAdapter invariant 7's requirement that request.metadata round-trip unchanged, so the divergence is deliberate. Callers who need OpenAI's request id for a support ticket should pass their own opts[:request_id] and correlate on that. Binding on 20.5 / 20.6.

Error-struct hygiene

%ALLM.Error.EmbeddingAdapterError{} derives Jason.Encoder and is commonly logged and persisted, so this adapter never copies a raw response body, a request header, or any Authorization value into :cause, :metadata, or :message. Provider error messages pass through a redactor that replaces key-shaped tokens with [REDACTED] — OpenAI echoes a prefix of the offending key back in its 401 text.

Retry integration

HTTP-error closures return {:retry, delay_ms, error} for 429 (honouring Retry-After), 5xx, timeouts, and transport failures; ALLM.Retry.run/3 is wrapped around each attempt. The closure returns real reason atoms rather than swapping in HTTP status codes, so for :rate_limited, :provider_unavailable, and :network_error the façade's widened retry_on list is what decides — none of them appears in this adapter's own opts[:retry] policy, which defaults to :default.

:timeout is the documented exception: it is a member of both lists, so through ALLM.embed/3 the adapter's inner ALLM.Retry.run/3 and the façade's outer one both retry it and the attempt budgets multiply (3 × 3 = 9 HTTP attempts at the default policy, against 3 for every other retryable reason). A direct embed/2 call takes the inner loop only and makes 3. ALLM.Providers.OpenAI.Images has the byte-identical shape through ALLM.generate_image/3, so this is a pre-existing library-wide characteristic rather than an embeddings one; it is tracked in ASKS.md and binding on 20.5 / 20.6 — do not "fix" it per-adapter, because the correction has to land in the façade and both image adapters at once.

Test-injection escape hatch

embed/2 honours opts[:adapter_opts][:embedding_script] as a documented test-only short-circuit: when the key is present, the call delegates to ALLM.Providers.FakeEmbeddings.embed/2 BEFORE any pre-flight gate runs and returns its result verbatim. This is what lets the injectable ALLM.EmbeddingAdapter conformance suite drive a real adapter without an HTTP stub library.

The switch keys on the presence of that per-call key and nothing else — no environment variable, no application config, no :persistent_term — so it stays confined to an explicit argument. Setting adapter_opts already implies full control of the call. Production callers do not populate it.

prepare_request/2 deliberately does NOT delegate under the same key: a scripted response has no Req.Request analogue, so it returns a stub error instead.

Summary

Functions

Execute a text-embedding request synchronously against OpenAI.

Return the maximum number of inputs OpenAI accepts in one /v1/embeddings call.

Return an unfired Req.Request configured exactly as embed/2 would fire it, for callers who need to add headers, middleware, or their own retry wrapper before dispatch.

Functions

embed(request, opts)

Execute a text-embedding request synchronously against OpenAI.

Returns {:ok, %ALLM.EmbeddingResponse{}} or {:error, %ALLM.Error.EmbeddingAdapterError{}}; every HTTP-shaped failure converts, including transport errors. The one documented exception is ALLM.Keys.fetch!/2, which raises %ALLM.Error.EngineError{reason: :missing_key} by design and is not rescued here — the three pre-flight gates all run ahead of it, so a request rejected pre-flight never needs a key.

See the module documentation for the gate order, the wire-field map, the no-injected-defaults policy for a nil :model, and the adapter_opts[:embedding_script] test-injection short-circuit.

Examples

iex> e = ALLM.Embedding.new(vector: [0.1, 0.2])
iex> req = ALLM.EmbeddingRequest.new(input: ["a kestrel"])
iex> opts = [adapter_opts: [embedding_script: [{:ok, [e]}]]]
iex> {:ok, resp} = ALLM.Providers.OpenAI.Embeddings.embed(req, opts)
iex> ALLM.EmbeddingResponse.vectors(resp)
[[0.1, 0.2]]

iex> req = ALLM.EmbeddingRequest.new(input: [])
iex> {:error, err} = ALLM.Providers.OpenAI.Embeddings.embed(req, [])
iex> err.reason
:invalid_request

max_batch_size()

@spec max_batch_size() :: pos_integer()

Return the maximum number of inputs OpenAI accepts in one /v1/embeddings call.

Per-module and constant, not per-model. Note the separate 300,000-token per-request cap described in the module documentation, which can be hit well below this number.

Examples

iex> ALLM.Providers.OpenAI.Embeddings.max_batch_size
2048

prepare_request(request, opts)

@spec prepare_request(
  ALLM.EmbeddingRequest.t(),
  keyword()
) :: {:ok, Req.Request.t()} | {:error, ALLM.Error.EmbeddingAdapterError.t()}

Return an unfired Req.Request configured exactly as embed/2 would fire it, for callers who need to add headers, middleware, or their own retry wrapper before dispatch.

The pre-flight gates run first, so this is defined only for a request whose input is non-empty and no longer than max_batch_size/0.

Under opts[:adapter_opts][:embedding_script] this returns a stub error rather than delegating to ALLM.Providers.FakeEmbeddings — a scripted response has no Req.Request analogue. That asymmetry with embed/2 is deliberate.

Examples

iex> req = ALLM.EmbeddingRequest.new(input: ["hi"], model: "text-embedding-3-small")
iex> {:ok, http} = ALLM.Providers.OpenAI.Embeddings.prepare_request(req, api_key: "sk-x")
iex> URI.to_string(http.url)
"https://api.openai.com/v1/embeddings"