Result of an embedding request, with the provenance needed to use it safely.
{:ok, result} = ExAgent.embed(provider, ["hello", "world"], task: :retrieval_document)
result.vectors #=> [[0.01, ...], [0.02, ...]]Store the provenance alongside every vector
Embedding spaces are model-scoped. Vectors from gemini-embedding-001 and
gemini-embedding-2 are not comparable - mixing them silently degrades
retrieval rather than failing, and the only fix is a full re-embed. The same
applies to a change in :dimensions or :task.
Persist :model, :dimensions, and :task next to the vector itself, so a
later model change is a detectable migration rather than a quiet regression.
Tasks belong to the provider
"What is this embedding for" is asked in incompatible ways, and the vocabulary
belongs to the model, not to this library: Gemini's taskType is a closed
enum of eight values, Jina v5 takes four task names plus a separate
prompt_name, and OpenAI has no task field at all. There is no portable middle
- a shared vocabulary either loses the distinctions a model actually makes or invents ones it does not.
So each provider declares its own atoms and rejects anything outside them:
ExAgent.embedding_tasks(gemini) #=> [:retrieval_query, :retrieval_document, ...]
ExAgent.embedding_tasks(jina) #=> [:retrieval, :text_matching, :clustering, :classification]A task the provider does not know is an error naming the ones it does, never a field quietly dropped: a dropped task returns HTTP 200 with plausible floats that land in a vector store and degrade retrieval forever, with no later signal.
Extra model arguments
:args forwards key-value pairs into the request body for parameters this
library does not model - Jina's prompt_name, OpenAI's encoding_format:
ExAgent.embed(jina, chunks, task: :retrieval, args: [prompt_name: :document])Each provider validates them against what its own endpoint accepts and rejects unknown keys, so a typo fails loudly instead of being ignored by the server.
Summary
Types
Extra request-body parameters, validated by the provider before being sent.
What to pass as :task: an atom from the provider's own vocabulary.
Functions
Cosine similarity between two vectors, in -1.0..1.0.
Scales a vector to unit length.
Normalizes :args into a keyword list, rejecting anything else.
Types
Extra request-body parameters, validated by the provider before being sent.
@type task() :: atom()
What to pass as :task: an atom from the provider's own vocabulary.
Discover it with ExAgent.embedding_tasks/1. There is deliberately no shared
vocabulary and no verbatim-string escape hatch - a task string an endpoint does
not recognize is accepted with a 200 and quietly wrong vectors.
Functions
Cosine similarity between two vectors, in -1.0..1.0.
Returns 0.0 if either vector is all zeros. Raises when the vectors differ in
length - that almost always means they came from different models or
dimension settings, which is a bug rather than a value worth computing.
Examples
iex> ExAgent.Embeddings.cosine_similarity([1.0, 0.0], [1.0, 0.0])
1.0
iex> ExAgent.Embeddings.cosine_similarity([1.0, 0.0], [0.0, 1.0])
0.0Scales a vector to unit length.
Some providers return non-unit vectors when the output is truncated to fewer dimensions; cosine similarity assumes unit length, so those must be normalized before use. A zero vector is returned unchanged rather than dividing by zero.
Examples
iex> ExAgent.Embeddings.l2_normalize([3.0, 4.0])
[0.6, 0.8]
iex> ExAgent.Embeddings.l2_normalize([0.0, 0.0])
[0.0, 0.0]Normalizes :args into a keyword list, rejecting anything else.
Providers call this before validating against their own allowlist, so :args
can be given as either a keyword list or a map.