@spec available?() :: boolean()

Whether this build has the LiteRT API at all.

It is a build option and it is off by default, so on an ordinary build every other function here answers {:error, "the LiteRT API was not compiled into this build..."}. Asking this first is cheaper than finding out from a call that was meant to do something.

@spec controlling_process(reference()) ::
  {:ok, pid()} | :undefined | {:error, String.t()}

Which process the model belongs to, or :undefined if it is unclaimed.

An unclaimed model is open to every process. Claiming one is controlling_process/2.

@spec controlling_process(reference(), pid()) :: :ok | {:error, String.t()}

Hand the model to a process, after which no other process may use it.

The claim is dropped when that process dies, so a crash does not strand the model.

Raising version of controlling_process/2.

@spec environment() :: {:ok, reference()} | {:error, String.t()}

A LiteRT environment, which the accelerator plugins are loaded into.

One is enough for any number of models and it has to outlive them.

@spec environment(String.t()) :: {:ok, reference()} | {:error, String.t()}

An environment that looks in runtime_library_dir for accelerator plugins.

Without a directory the plugins are searched for relative to nothing, which is the usual reason a GPU accelerator silently does not load.

Raising version of environment/0.

Raising version of environment/1.

@spec fully_accelerated(reference()) :: {:ok, boolean()} | {:error, String.t()}

As fully_accelerated?/1, but reports why it could not be answered.

Raising version of fully_accelerated/1.

@spec fully_accelerated?(reference()) :: boolean()

Whether the accelerator took the whole graph.

A partly accelerated model pays for every crossing between the accelerator and the CPU, and is often slower than the CPU alone, so a false here is worth acting on rather than ignoring.

@spec io_sizes(reference()) ::
  {:ok, {[non_neg_integer()], [non_neg_integer()]}} | {:error, String.t()}

The byte size of each input and output tensor, as {inputs, outputs}.

Raising version of io_sizes/1.

@spec new(reference(), String.t()) :: {:ok, reference()} | {:error, String.t()}

Compile a model with the default options: CPU, no profiling.

@spec new(reference(), String.t(), opts()) ::
  {:ok, reference()} | {:error, String.t()}

Compile a model.

Options

• :accelerators. What to run on, in order of preference, e.g. [:cpu, :gpu]. Naming an accelerator asks for it; whether it was used is fully_accelerated?/1. Defaults to [:cpu].
• :precision. :default, :fp16, :fp32 or :fp16_with_fp32_accum.
• :profile. Record per-operator timings, readable with profile/1 and summarise_profile/1. Off by default because it is not free.
• :signature. Which signature to compile for, by index or by name.
• :max_model_bytes. Refuse a model file larger than this.

Raising version of new/2.

Raising version of new/3.

@spec pending_events(reference()) :: {:ok, non_neg_integer()} | {:error, String.t()}

How many profiling events are waiting, without reading them.

Zero for a model compiled without profile: true. This is what sizes the copy profile/2 has to make, so it is the number to look at before calling it on a memory-constrained target.

Raising version of pending_events/1.

@spec platform_support() ::
  {:ok, %{required(atom()) => boolean()}} | {:error, String.t()}

Which buffer kinds this platform can reach, e.g. %{metal: true, opencl: false}.

Answers {:error, reason} on a build without the LiteRT API; available?/0 is the question to ask first.

@spec profile(reference()) :: {:ok, [event()]} | {:error, String.t()}

Every profiling event recorded so far, oldest first.

@spec profile(reference(), non_neg_integer()) ::
  {:ok, [event()]} | {:error, String.t()}

The most recent limit profiling events, or all of them when limit is zero.

limit bounds the events returned, not the reading: LiteRT will not hand over part of a backlog, so every call copies whatever pending_events/1 reports. That is about 109 MiB for a full buffer, which is nothing on a workstation and fatal on a board with 256 MB.

Raising version of profile/1.

Raising version of profile/2.

@spec reset_profile(reference()) :: :ok | {:error, String.t()}

Forget the events recorded so far and keep recording.

Raising version of reset_profile/1.

@spec run(reference(), [binary()]) :: {:ok, [binary()]} | {:error, String.t()}

Run the model over a list of input binaries, one per input tensor.

The sizes each input has to be are io_sizes/1, and a wrong size is refused rather than read past.

Raising version of run/2.

@spec run_with_metrics(reference(), [binary()]) ::
  {:ok, {[binary()], [{binary(), metric_value()}]}} | {:error, String.t()}

Run the model and collect whatever counters the accelerator reports.

@spec run_with_metrics(reference(), [binary()], non_neg_integer()) ::
  {:ok, {[binary()], [{binary(), metric_value()}]}} | {:error, String.t()}

Run with metrics collection bracketing the inference.

Usually the counters come back empty. Filling them in is the accelerator's job, through two entries of its definition that are allowed to be null, so an empty list means nobody offered anything rather than that something went wrong. Use profile/1 for timings; this is for counters a backend chooses to expose.

Raising version of run_with_metrics/2.

Raising version of run_with_metrics/3.

@spec signatures(reference(), String.t()) :: {:ok, [binary()]} | {:error, String.t()}

The names of a model file's signatures, without compiling it.

A model with no named signature reports the one default signature LiteRT gives it, so the list is never empty.

Raising version of signatures/2.

@spec summarise_profile(reference()) ::
  {:ok, [summary_entry()]} | {:error, String.t()}

Per-operator totals over every run since the last reset, slowest first.

Only operator events are folded in. The enclosing Invoke, tensor allocation and LiteRT's own buffer handling are events too, and adding them together would count the operators twice, so they stay in profile/1 alone.

Raising version of summarise_profile/1.