barrel_embed_clip (barrel_embed v2.3.0)

View Source

CLIP image/text embedding provider

Uses CLIP (Contrastive Language-Image Pre-training) models for cross-modal embeddings. Both images and text are encoded into the same vector space, enabling image-text similarity search.

Dependencies (transformers, torch, pillow) are installed automatically in the managed venv on first use.

Configuration

   Config = #{
       model => "openai/clip-vit-base-patch32",   %% Model name (default)
       python => "python3",                       %% Python executable (default)
       timeout => 120000                          %% Timeout in ms (default)
   }.

CLIP enables searching images with text queries and vice versa:

   %% Embed an image
   {ok, ImgVec} = embed_image(ImageBase64, Config),
  
   %% Embed a text query (in same space!)
   {ok, TextVec} = embed(<<"a photo of a cat">>, Config),
  
   %% Now you can compare ImgVec and TextVec with cosine similarity

Supported Models

- "openai/clip-vit-base-patch32" - Default, 512 dimensions, fast - "openai/clip-vit-base-patch16" - 512 dimensions, higher quality - "openai/clip-vit-large-patch14" - 768 dimensions, best quality - "laion/CLIP-ViT-B-32-laion2B-s34B-b79K" - 512 dims, LAION trained

Use Cases

- Image search with text queries - Finding similar images - Multi-modal content retrieval - Zero-shot image classification

Summary

Functions

available(Config)

Check if provider is available.

dimension(Config)

Get dimension for this provider.

embed(Text, Config)

Generate text embedding (for cross-modal search). Text embeddings are in the same space as image embeddings.

embed_batch(Texts, Config)

Generate text embeddings for batch.

embed_image(ImageBase64, Config)

Generate embedding for a single image. Image should be base64-encoded.

embed_image_batch(Images, Config)

Generate embeddings for multiple images. Images should be base64-encoded.

init(Config)

Initialize the provider.

name()

Provider name.

Functions

available(Config)

-spec available(map()) -> boolean().

Check if provider is available.

dimension(Config)

-spec dimension(map()) -> pos_integer().

Get dimension for this provider.

embed(Text, Config)

-spec embed(binary(), map()) -> {ok, [float()]} | {error, term()}.

Generate text embedding (for cross-modal search). Text embeddings are in the same space as image embeddings.

embed_batch(Texts, Config)

-spec embed_batch([binary()], map()) -> {ok, [[float()]]} | {error, term()}.

Generate text embeddings for batch.

embed_image(ImageBase64, Config)

-spec embed_image(binary(), map()) -> {ok, [float()]} | {error, term()}.

Generate embedding for a single image. Image should be base64-encoded.

embed_image_batch(Images, Config)

-spec embed_image_batch([binary()], map()) -> {ok, [[float()]]} | {error, term()}.

Generate embeddings for multiple images. Images should be base64-encoded.

init(Config)

-spec init(map()) -> {ok, map()} | {error, term()}.

Initialize the provider.

name()

-spec name() -> atom().

Provider name.