Planck.AI.Models.CustomOpenAI (Planck.AI v0.1.5)

Factory and discovery for OpenAI-compatible endpoints (e.g. NVIDIA, Together, vLLM, LM Studio).

Unlike cloud provider modules, this queries the running server at call time because the available models depend on the specific endpoint.

Examples

iex> Planck.AI.Models.CustomOpenAI.model("meta/llama-3.1-8b-instruct",
...>   identifier: "nvidia",
...>   base_url: "https://integrate.api.nvidia.com/v1",
...>   context_window: 128_000
...> )
%Planck.AI.Model{provider: :custom_openai, identifier: "nvidia", ...}

Summary

Functions

all(opts \\ [])

Discovers models from an OpenAI-compatible /models endpoint.

model(id, opts \\ [])

Builds a Planck.AI.Model for a custom OpenAI-compatible endpoint.

Functions

all(opts \\ [])

@spec all(keyword()) :: [Planck.AI.Model.t()]

Discovers models from an OpenAI-compatible /models endpoint.

Options

:base_url — base URL of the server (required).
:api_key — bearer token sent as Authorization: Bearer <key>.
:identifier — name used to identify this endpoint (e.g. "nvidia"). Stored on each returned model.
:context_window — default context window for discovered models. Defaults to 4_096.
:max_tokens — default max tokens. Defaults to 2_048.

model(id, opts \\ [])

@spec model(
  String.t(),
  keyword()
) :: Planck.AI.Model.t()

Builds a Planck.AI.Model for a custom OpenAI-compatible endpoint.

Options

:identifier — name used to identify this endpoint (e.g. "nvidia").
:base_url — base URL of the server.
:api_key — bearer token for authentication.
:name — display name. Defaults to id.
:context_window — context window size. Defaults to 4_096.
:max_tokens — max tokens to generate. Defaults to 2_048.
:supports_thinking — whether the model supports thinking blocks. Defaults to false.
:input_types — list of supported input modalities. Defaults to [:text].
:default_opts — inference parameters applied on every call. Defaults to [].