AgentSea.Embedder.Bumblebee (agentsea_bumblebee v0.1.0)

Copy Markdown

In-process text embedder backed by a Hugging Face model via Bumblebee + Nx — the production AgentSea.Embedder (no external embedding API).

Build a serving once (it loads the model), then pass it on each call:

serving = AgentSea.Embedder.Bumblebee.serving("sentence-transformers/all-MiniLM-L6-v2")
{:ok, vectors} = AgentSea.Embedder.Bumblebee.embed(["hello", "world"], serving: serving)

Runs on Nx's pure-Elixir backend out of the box. For real throughput, enable EXLA — add {:exla, "~> 0.9"} and configure either globally:

config :nx, default_backend: EXLA.Backend

or per-serving (compiled defn), via app config that serving/2 merges in:

config :agentsea_bumblebee,
  serving_options: [
    defn_options: [compiler: EXLA],
    compile: [batch_size: 1, sequence_length: 256]
  ]

The serving call is injectable (:run) so the embedding plumbing is testable without loading a model.

Summary

Functions

Build an Nx.Serving for a text-embedding model. Downloads the model on first use (network), so this is a runtime/setup call, not used in unit tests.

Serving options from app config (e.g. an EXLA compiler), merged into serving/2 with per-call opts taking precedence. See the module doc for enabling EXLA.

Functions

serving(model_id \\ "sentence-transformers/all-MiniLM-L6-v2", opts \\ [])

Build an Nx.Serving for a text-embedding model. Downloads the model on first use (network), so this is a runtime/setup call, not used in unit tests.

serving_options()

Serving options from app config (e.g. an EXLA compiler), merged into serving/2 with per-call opts taking precedence. See the module doc for enabling EXLA.