Nous.Providers.LMStudio (nous v0.16.3)

View Source

LM Studio local provider implementation.

LM Studio provides a local OpenAI-compatible API server for running models locally. By default it runs on http://localhost:1234/v1.

Configuration

No API key is required for local usage. Configure the base URL if needed:

config :nous, :lmstudio,
  base_url: "http://localhost:1234/v1"

Or use environment variable:

export LMSTUDIO_BASE_URL="http://localhost:1234/v1"

Usage

# Via Model.parse
model = Nous.Model.parse("lmstudio:my-local-model")

# Direct provider usage
{:ok, response} = Nous.Providers.LMStudio.chat(%{
  "model" => "my-local-model",
  "messages" => [%{"role" => "user", "content" => "Hello"}]
})

Features

LM Studio supports:

  • OpenAI-compatible chat completions
  • Streaming responses
  • Tool/function calling (model-dependent)
  • Various open-source models (Llama, Mistral, etc.)

Summary

Functions

api_key(opts \\ [])

Get the API key from options, environment, or application config.

base_url(opts \\ [])

Get the base URL from options, application config, or default.

count_tokens(messages)

Count tokens in messages (rough estimate).

request(model, messages, settings)

High-level request with message conversion, telemetry, and error wrapping.

request_stream(model, messages, settings)

High-level streaming request with message conversion and telemetry.

Functions

api_key(opts \\ [])

@spec api_key(keyword()) :: String.t() | nil

Get the API key from options, environment, or application config.

Lookup order:

  1. :api_key option passed directly
  2. Environment variable (LMSTUDIO_API_KEY)
  3. Application config: config :nous, lmstudio, api_key: "..."

base_url(opts \\ [])

@spec base_url(keyword()) :: String.t()

Get the base URL from options, application config, or default.

Lookup order:

  1. :base_url option passed directly
  2. Application config: config :nous, lmstudio, base_url: "..."
  3. Default: http://localhost:1234/v1

count_tokens(messages)

@spec count_tokens(list()) :: integer()

Count tokens in messages (rough estimate).

Override this in your provider for more accurate counting.

request(model, messages, settings)

High-level request with message conversion, telemetry, and error wrapping.

Default implementation that:

  1. Converts messages to provider format
  2. Builds request params
  3. Calls chat/2
  4. Parses response
  5. Emits telemetry events
  6. Wraps errors

request_stream(model, messages, settings)

High-level streaming request with message conversion and telemetry.