LLMProxy (llm_proxy v0.1.0)

Copy Markdown View Source

Elixir-native LLM gateway for in-process, ReqLLM, SafeRPC, and HTTP calls.

LLMProxy applies model routing, provider fallback, API-key governance, usage accounting, and tracing through one provider execution boundary. It can run inside an existing application or as a standalone OpenAI-compatible service.

Summary

Functions

chat(messages, opts \\ [])

Calls the proxy in-process using ReqLLM messages or a plain prompt.

chat(request, meta, state)

Run a chat completion through LLMProxy.

models(payload, meta, state)

List available models.

status(payload, meta, state)

Return service status.

Functions

chat(messages, opts \\ [])

Calls the proxy in-process using ReqLLM messages or a plain prompt.

Pass either :actor with %LLMProxy.Actor{} or :api_key with an existing LLMProxy API key schema/map so quota and usage accounting can run.

chat(request, meta, state)

@spec chat(LLMProxy.Protocol.Request.t(), map(), term()) ::
  {:ok, LLMProxy.Response.t()} | {:error, term()}

Run a chat completion through LLMProxy.

models(payload, meta, state)

@spec models(map(), map(), term()) :: {:ok, [map()]}

List available models.

status(payload, meta, state)

@spec status(map(), map(), term()) :: {:ok, map()}

Return service status.