View Source TFLiteElixir.LiteRT.CompiledModel.Isolated (tflite_elixir v1.0.0-rc4)
A compiled model on a node of its own, so that a crash inside the runtime costs one node instead of yours.
Everything under this binding is native code, and native code that segfaults takes the whole VM with it regardless of how carefully the Elixir above it is written. Where that is unacceptable, this puts the model on a separate BEAM node and forwards calls to it:
{:ok, model} = TFLiteElixir.LiteRT.CompiledModel.Isolated.start_link(
model_path: path, accelerators: [:cpu, :gpu])
{:ok, outputs} = TFLiteElixir.LiteRT.CompiledModel.Isolated.run(model, inputs)The far side is an ordinary TFLiteElixir.LiteRT.CompiledModel.Server, so the
calls are the same ones; only the wire is different. When the node dies the
calls answer {:error, "the isolated model is no longer there"} rather than
hanging or taking the caller down.
What it costs
Every input and every output crosses the distribution link, which for image sized tensors is not free, and the node takes a moment to start. Reach for it when a crash must not be fatal, not by default.
Distribution is started if it is not already up, because a library should not insist the caller arranged that in advance.
Summary
Functions
Whether the accelerator took the whole graph.
Raising version of fully_accelerated/1.
As fully_accelerated/1, answering false rather than an error.
The byte size of each input and output tensor, as {inputs, outputs}.
Raising version of io_sizes/1.
The node the model is running on.
Raising version of node_of/1.
How many profiling events are waiting, without reading them.
Raising version of pending_events/1.
Every profiling event recorded so far.
The most recent limit profiling events, or all of them when zero.
Raising version of profile/1.
Raising version of profile/2.
Forget the events recorded so far and keep recording.
Raising version of reset_profile/1.
Run the model over a list of input binaries.
Run the model, waiting at most timeout.
Raising version of run/2.
Raising version of run/3.
Run the model and collect whatever counters the accelerator reports.
As run_with_metrics/2, at a given detail level.
As run_with_metrics/3, waiting at most timeout.
Raising version of run_with_metrics/2.
Raising version of run_with_metrics/3.
Start one outside a supervision tree.
As start/1, with options for the gen_server itself.
Start a model on a node of its own, linked to the caller.
As start_link/1, with options for the gen_server itself.
Stop the model and the node it is on.
Per-operator totals over every run since the last reset, slowest first.
Raising version of summarise_profile/1.
Run a function against the compiled model, on the node that owns it.
As with/2, waiting at most timeout.
Types
@type opts() :: [ model_path: String.t(), runtime_library_dir: String.t(), accelerators: [TFLiteElixir.LiteRT.CompiledModel.accelerator()], precision: TFLiteElixir.LiteRT.CompiledModel.precision(), profile: boolean(), signature: non_neg_integer() | String.t(), max_model_bytes: non_neg_integer(), max_queue: non_neg_integer(), peer_args: [String.t()] ]
Functions
Whether the accelerator took the whole graph.
Raising version of fully_accelerated/1.
As fully_accelerated/1, answering false rather than an error.
@spec io_sizes(pid()) :: {:ok, {[non_neg_integer()], [non_neg_integer()]}} | {:error, String.t()}
The byte size of each input and output tensor, as {inputs, outputs}.
Raising version of io_sizes/1.
The node the model is running on.
Raising version of node_of/1.
@spec pending_events(pid()) :: {:ok, non_neg_integer()} | {:error, String.t()}
How many profiling events are waiting, without reading them.
Raising version of pending_events/1.
@spec profile(pid()) :: {:ok, [TFLiteElixir.LiteRT.CompiledModel.event()]} | {:error, String.t()}
Every profiling event recorded so far.
@spec profile(pid(), non_neg_integer()) :: {:ok, [TFLiteElixir.LiteRT.CompiledModel.event()]} | {:error, String.t()}
The most recent limit profiling events, or all of them when zero.
Raising version of profile/1.
Raising version of profile/2.
Forget the events recorded so far and keep recording.
Raising version of reset_profile/1.
Run the model over a list of input binaries.
Run the model, waiting at most timeout.
Raising version of run/2.
Raising version of run/3.
@spec run_with_metrics(pid(), [binary()]) :: {:ok, {[binary()], [{binary(), TFLiteElixir.LiteRT.CompiledModel.metric_value()}]}} | {:error, String.t()}
Run the model and collect whatever counters the accelerator reports.
@spec run_with_metrics(pid(), [binary()], non_neg_integer()) :: {:ok, {[binary()], [{binary(), TFLiteElixir.LiteRT.CompiledModel.metric_value()}]}} | {:error, String.t()}
As run_with_metrics/2, at a given detail level.
@spec run_with_metrics(pid(), [binary()], non_neg_integer(), timeout()) :: {:ok, {[binary()], [{binary(), TFLiteElixir.LiteRT.CompiledModel.metric_value()}]}} | {:error, String.t()}
As run_with_metrics/3, waiting at most timeout.
Raising version of run_with_metrics/2.
Raising version of run_with_metrics/3.
Start one outside a supervision tree.
As start/1, with options for the gen_server itself.
Start a model on a node of its own, linked to the caller.
:model_path is required. The rest are what
TFLiteElixir.LiteRT.CompiledModel.new/3 and
TFLiteElixir.LiteRT.CompiledModel.Server.start_link/3 take, plus:
:peer_args. Extra arguments for the node itself, e.g.["+sbwt", "none"].
As start_link/1, with options for the gen_server itself.
@spec stop(pid()) :: :ok
Stop the model and the node it is on.
@spec summarise_profile(pid()) :: {:ok, [TFLiteElixir.LiteRT.CompiledModel.summary_entry()]} | {:error, String.t()}
Per-operator totals over every run since the last reset, slowest first.
Raising version of summarise_profile/1.
Run a function against the compiled model, on the node that owns it.
The callback is sent to that node and applied there, so it must return something worth sending back: a value, not a handle to something local to it.
@spec with(pid(), (reference() -> result), timeout()) :: result | {:error, String.t()} when result: term()
As with/2, waiting at most timeout.