View Source Ollama.API (Ollama v0.1.0)
Client module for interacting with the Ollama API.
Currently supporting all Ollama API endpoints except pushing models (/api/push
),
which is coming soon.
Usage
Assuming you have Ollama running on localhost, and that you have installed a
model, immediately trying the completion/4
and chat/4
functions should be very
straightforward.
iex> api = Ollama.API.new
iex> Ollama.API.completion(api, "llama2", "Why is the sky blue?")
{:ok, %{"response" => "The sky is blue because it is the color of the sky."}}
Summary
Types
Stream handler callback function
Chat message
Model name, in the format <model name>:<tag>
API function response
Client struct
Functions
Generates the next message in a chat using the specified model. Optionally streamable.
Checks a blob exists in ollama by its digest or binary data.
Generates a completion for the given prompt using the specified model. Optionally streamable.
Creates a model with another name from an existing model.
Creates a blob from its binary data.
Creates a model using the given name and model file. Optionally streamable.
Deletes a model and its data.
Generate embeddings from a model for the given prompt.
Lists all models that Ollama has available.
Creates a new API client with the provided URL. If no URL is given, it
defaults to "http://localhost:11434/api"
.
Downloads a model from the ollama library. Optionally streamable.
Shows all information for a specific model.
Types
@type handle_stream() :: (map() -> nil)
Stream handler callback function
@type message() :: map()
Chat message
A chat message is a map/0
with the following fields:
role
- The role of the message, eithersystem
,user
orassistant
.content
- The content of the message.images
- (optional) List of Base64 encoded images (for multimodal models only).
@type model() :: String.t()
Model name, in the format <model name>:<tag>
API function response
@type t() :: %Ollama.API{req: Req.Request.t()}
Client struct
Functions
Generates the next message in a chat using the specified model. Optionally streamable.
Message structure
A list of chat messages must be passed to this request. Each message has the following fields:
role
- The role of the message, eithersystem
,user
orassistant
.content
- The content of the message.images
- (optional) List of Base64 encoded images (for multimodal models only).
Options
The following options are accepted:
:options
- Additional advanced model parameters:template
- Prompt template, overriding the model default:stream
- A callback to handle streaming response chunks.
Examples
iex> messages = [
...> %{role: "system", content: "You are a helpful assistant."},
...> %{role: "user", content: "Why is the sky blue?"},
...> %{role: "assistant", content: "Due to rayleigh scattering."},
...> %{role: "user", content: "How is that different than mie scattering?"},
...> ]
# Passing a callback to the :stream option initiates a streaming request.
iex> Ollama.API.chat(api, "llama2", messages, stream: fn data ->
...> IO.inspect(data) # %{"message" => %{"role" => "assistant", "content" => "Mie"}}
...> end)
{:ok, ""}
# Without the :stream option initiates a standard request
iex> Ollama.API.chat(api, "llama2", messages)
{:ok, %{"message" => %{
"role" => "assistant",
"content" => "Mie scattering affects all wavelengths similarly, while Rayleigh favors shorter ones."
}, ...}}
@spec check_blob(t(), Ollama.Blob.digest() | binary()) :: response()
Checks a blob exists in ollama by its digest or binary data.
Examples
iex> Ollama.API.check_blob(api, "sha256:fe938a131f40e6f6d40083c9f0f430a515233eb2edaa6d72eb85c50d64f2300e")
{:ok, true}
iex> Ollama.API.check_blob(api, "this should not exist")
{:ok, false}
Generates a completion for the given prompt using the specified model. Optionally streamable.
Options
The following options are accepted:
:images
- A list of Base64 encoded images to be included with the prompt (for multimodal models only).:options
- Additional advanced model parameters.:system
- System prompt, overriding the model default.:template
- Prompt template, overriding the model default.:context
- The context parameter returned from a previousf:completion/4
call (enabling short conversational memory).:stream
- A callback to handle streaming response chunks.
Examples
# Passing a callback to the :stream option initiates a streaming request.
iex> Ollama.API.completion(api, "llama2", "Why is the sky blue?", stream: fn data ->
...> IO.inspect(data) # %{"response" => "The"}
...> end)
{:ok, ""}
# Without the :stream option initiates a standard request
iex> Ollama.API.completion(api, "llama2", "Why is the sky blue?")
{:ok, %{"response": "The sky is blue because it is the color of the sky.", ...}}
Creates a model with another name from an existing model.
Example
iex> Ollama.API.copy_model(api, "llama2", "llama2-backup")
{:ok, true}
Creates a blob from its binary data.
Example
iex> Ollama.API.create_blob(api, modelfile)
{:ok, true}
Creates a model using the given name and model file. Optionally streamable.
Any dependent blobs reference in the modelfile, such as FROM
and ADAPTER
instructions, must exist first. See check_blob/2
and create_blob/2
.
Options
The following options are accepted:
:stream
- A callback to handle streaming response chunks.
Example
iex> modelfile = "FROM llama2\nSYSTEM \"You are mario from Super Mario Bros.\""
iex> Ollama.API.create_model(api, "mario", modelfile, stream: fn data ->
...> IO.inspect(data) # %{"status" => "reading model metadata"}
...> end)
{:ok, ""}
Deletes a model and its data.
Example
iex> Ollama.API.delete_model(api, "llama2")
{:ok, true}
Generate embeddings from a model for the given prompt.
Example
iex> Ollama.API.embeddings(api, "llama2", "Here is an article about llamas...")
{:ok, %{"embedding" => [
0.5670403838157654, 0.009260174818336964, 0.23178744316101074, -0.2916173040866852, -0.8924556970596313,
0.8785552978515625, -0.34576427936553955, 0.5742510557174683, -0.04222835972905159, -0.137906014919281
]}}
Lists all models that Ollama has available.
Example
iex> Ollama.API.list_models(api)
{:ok, %{"models" => [
%{"name" => "codellama:13b", ...},
%{"name" => "llama2:latest", ...},
]}}
Creates a new API client with the provided URL. If no URL is given, it
defaults to "http://localhost:11434/api"
.
Examples
iex> api = Ollama.API.new("https://ollama.service.ai:11434")
%Ollama.API{}
Downloads a model from the ollama library. Optionally streamable.
Options
The following options are accepted:
:stream
- A callback to handle streaming response chunks.
Example
iex> Ollama.API.pull_model(api, "llama2", stream: fn data ->
...> IO.inspect(data) # %{"status" => "pulling manifest"}
...> end)
{:ok, ""}
Shows all information for a specific model.
Example
iex> Ollama.API.show_model(api, "llama2")
{:ok, %{
"details" => %{
"families" => ["llama", "clip"],
"family" => "llama",
"format" => "gguf",
"parameter_size" => "7B",
"quantization_level" => "Q4_0"
},
"modelfile" => "...",
"parameters" => "...",
"template" => "..."
}}