Modules
Candil — LLM inference and model management for Elixir.
A minimal ReAct-style agent that uses tools.
OTP application for Candil.
Behaviour that abstracts the local/remote differences between LLM backends. A backend is responsible for
Candil.Backend implementation for local llama.cpp / llama-server.
Candil.Backend implementation for OpenAI-compatible HTTP APIs.
Registry of in-flight LLM generations, keyed by a cancel_ref/0.
Engine and model registry for Candil.Llm.
Reads, validates, and normalizes LLM/embedding provider configuration.
Conversation history management for Candil.Llm.
Context window management for conversation history.
Token estimation utilities for conversation context management.
Cost estimation for LLM API usage.
System capability detection for llama.cpp precompiled binary selection.
Embedding generation abstraction over multiple providers.
Local llama.cpp engine definition and lifecycle management.
Shared health-polling logic for engine GenServers.
Behaviour for custom engine launchers.
GenServer that supervises a single llama-server OS process.
GenServer for engines whose OS process is owned by something else.
Lightweight LRU pool for Candil.Engine instances.
Unified error types for Candil.
Shared HTTP client with circuit breaker, retry, and rate limiting for Candil.
Health checks for LLM/embedding providers.
Inference execution for Candil.
Download and installation utilities for llama.cpp binaries and GGUF models.
LLM model definition for Candil.
Remote LLM provider definition for Candil.
Global sliding-window rate limiter backed by ETS.
Shared request body builders for LLM providers.
Server-Sent Events (SSE) streaming for LLM inference.
Force the model to produce JSON matching a schema.
Telemetry events emitted by Candil during LLM inference.
Defines a callable tool that an LLM can invoke.
Tool calling for LLM responses.