-spec allocate_tensors(reference()) -> ok | {error, binary()}.

Allocate memory for tensors in the graph

-spec cancel(reference()) -> ok | {error, binary()}.

Ask an in-flight invoke/1 to stop.

Does not block and is safe to call from another process, which is the point: an invocation runs on a dirty scheduler and cannot otherwise be interrupted. Later invocations are unaffected. Requires enable_cancellation/1.

-spec controlling_process(reference()) -> {ok, pid()} | undefined | {error, binary()}.

Which process this interpreter belongs to, or undefined if it is shared.

The answer is read under the same lock every call takes, so while another process's call is in flight this is {error, Reason} rather than a wait.

-spec controlling_process(reference(), pid() | undefined) -> ok | {error, binary()}.

Give this interpreter to a process, after which no other process may use it.

tflite::Interpreter is not thread-safe and invoke/1 runs on a dirty scheduler, so two processes sharing one interpreter really do reach it on two OS threads. Measured on a real model, two processes taking turns badly got the wrong inference back 147 times out of 400 -- not a crash, just quietly somebody else's answer.

An interpreter starts out belonging to nobody, which is how they have always behaved, and calls from concurrent processes are refused only while they actually overlap. Naming a controlling process closes the remaining window: every other process is then refused outright, whether it overlaps or not.

Follows gen_tcp:controlling_process/2: while an interpreter belongs to nobody any process may take it, and once it belongs to someone only that process may hand it on. Pass undefined to give it back to nobody. A controlling process that dies releases it, since an interpreter has no equivalent of a socket being closed.

-spec enable_cancellation(reference()) -> ok | {error, binary()}.

Allow a running invoke/1 to be cancelled.

Has to be called before invoking. Without it cancel/1 is an error.

-spec execution_plan(reference()) -> [non_neg_integer()] | {error, binary()}.

Return the execution plan of the model.

Experimental interface, subject to change.

-spec get_allow_fp16_precision_for_fp32(reference()) -> {ok, boolean()} | {error, binary()}.

Whether float32 operations may be carried out in float16.

-spec get_input_name(reference(), non_neg_integer()) -> {ok, binary()} | {error, binary()}.

Get the name of the input tensor

Note that the index here means the index in the result list of inputs/1. For example, if inputs/1 returns [42, 314], then 0 should be passed here to get the name of tensor 42

-spec get_output_name(reference(), non_neg_integer()) -> {ok, binary()} | {error, binary()}.

Get the name of the output tensor

Note that the index here means the index in the result list of outputs/1. For example, if outputs/1 returns [42, 314], then 0 should be passed here to get the name of tensor 42

-spec get_signature_defs(reference()) -> {ok, map() | nil} | {error, binary()}.

Get SignatureDef map from the Metadata of a TfLite FlatBuffer buffer.

-spec get_signature_runner(reference(), binary() | list() | nil) ->
                              {ok, reference()} | {error, binary()}.

Get a runner for one of the model's signatures.

The runner addresses its tensors by name and belongs to this interpreter: it must not be used once the interpreter is gone. Passing a key the model does not declare is an error. Pass nil for the primary subgraph: the first signature that points at it, or a placeholder one when the model declares no signatures at all. See tflite_beam_signature_runner.

-spec get_subgraph_index_from_signature(reference(), binary() | list()) ->
                                           {ok, integer()} | {error, binary()}.

The subgraph a signature belongs to, or -1 for a key the model does not declare.

-spec input_tensor(reference(), non_neg_integer(), binary()) -> ok | {error, binary()}.

Fill data to the specified input tensor

Note: although we have typed_input_tensor available in C++, here what we really passed to the NIF is binary` data, therefore, Im not pretend that we have type information.

-spec inputs(reference()) -> {ok, [non_neg_integer()]} | {error, binary()}.

Get the list of input tensors.

return a list of input tensor id

-spec invoke(reference()) -> ok | {error, binary()}.

Run forwarding

-spec new() -> {ok, reference()} | {error, binary()}.

New interpreter

-spec new(list() | binary()) -> {ok, reference()} | {error, binary()}.

New interpreter with model filepath

-spec new_from_buffer(binary()) -> {ok, reference()} | {error, binary()}.

New interpreter with model buffer

-spec nodes_size(reference()) -> non_neg_integer() | {error, binary()}.

Return the number of ops in the model.

-spec output_tensor(reference(), non_neg_integer()) -> {ok, binary()} | {error, binary()}.

Get the data of the output tensor

Note that the index here means the index in the result list of outputs/1. For example, if outputs/1 returns [42, 314], then 0` should be passed here to get the name of tensor `42

-spec outputs(reference()) -> {ok, [non_neg_integer()]} | {error, binary()}.

Get the list of output tensors.

return a list of output tensor id

-spec predict(reference(), [binary()] | binary() | map()) -> [binary()] | {error, binary()}.

Fill input data to corresponding input tensor of the interpreter, call tflite_beam_interpreter:invoke/1 and return output tensor(s). fetch_outputs/2 reads each output with tflite_beam_tensor:to_binary/1, so what comes back is the bytes, not the records this used to promise, and an output that cannot be read fails the whole call rather than sitting in the list.

-spec release_non_persistent_memory(reference()) -> ok | {error, binary()}.

Release memory that is not needed between invocations.

Invoking again reallocates it, so this trades time for memory on devices short of the latter.

-spec reset_variable_tensors(reference()) -> ok | {error, binary()}.

Reset all variable tensors to zero.

tflite_beam:reset_variable_tensor/1 resets a single one.

-spec resize_input_tensor(reference(), integer(), [integer()] | tuple()) -> ok | {error, binary()}.

Change the dimensionality of a given input tensor.

Note that this is only acceptable for tensors that are inputs to the model, and allocate_tensors/1 has to be called again afterwards.

Dims is a list, or the tuple tflite_beam_tensor:shape/1 returns.

-spec resize_input_tensor_strict(reference(), integer(), [integer()] | tuple()) ->
                                    ok | {error, binary()}.

Change the dimensionality of a given input tensor, keeping the rank fixed.

Unlike resize_input_tensor/3, this only accepts dimensions that the model left unknown, so a tensor whose shape is fully fixed cannot be resized.

Dims is a list, or the tuple tflite_beam_tensor:shape/1 returns.

-spec set_allow_fp16_precision_for_fp32(reference(), boolean()) -> ok | {error, binary()}.

Allow or forbid carrying out float32 operations in float16.

Only has an effect on backends that can do it, and has to be set before the graph is prepared.

-spec set_inputs(reference(), [non_neg_integer()]) -> ok | {error, binary()}.

Provide a list of tensor indexes that are inputs to the model. Each index is bound check and this modifies the consistent_ flag of the interpreter.

-spec set_num_threads(reference(), integer()) -> ok | {error, binary()}.

Set the number of threads available to the interpreter.

As TfLite interpreter could internally apply a TfLite delegate by default (i.e. XNNPACK), the number of threads that are available to the default delegate should be set via InterpreterBuilder APIs as follows:

  {ok, Interpreter} = tflite_beam_interpreter:new(),
  {ok, Builder} = tflite_beam_interpreter_builder:new(Model, Resolver),
  tflite_beam_interpreter_builder:set_num_threads(Builder, NumThreads),
  tflite_beam_interpreter_builder:build(Builder, Interpreter)

NumThreads follows TfLite: -1 asks the runtime to choose, 0 means the same as 1, and anything below -1 is refused.

-spec set_outputs(reference(), [non_neg_integer()]) -> ok | {error, binary()}.

Provide a list of tensor indexes that are outputs to the model. Each index is bound check and this modifies the consistent_ flag of the interpreter.

-spec set_variables(reference(), [non_neg_integer()]) -> ok | {error, binary()}.

Provide a list of tensor indexes that are variable tensors. Each index is bound check and this modifies the consistent_ flag of the interpreter.

-spec signature_inputs(reference(), binary() | list()) -> {ok, map()} | {error, binary()}.

The inputs of the named signature, as a map of name to tensor index.

An empty map is returned for a key the model does not declare.

-spec signature_keys(reference()) -> [binary()] | {error, binary()}.

Returns list of all keys of different method signatures defined in the model.

WARNING: Experimental interface, subject to change

-spec signature_outputs(reference(), binary() | list()) -> {ok, map()} | {error, binary()}.

The outputs of the named signature, as a map of name to tensor index.

An empty map is returned for a key the model does not declare.

-spec subgraphs_size(reference()) -> {ok, non_neg_integer()} | {error, binary()}.

How many subgraphs the model has.

-spec tensor(reference(), non_neg_integer()) ->
                #tflite_beam_tensor{name :: binary(),
                                    index :: non_neg_integer(),
                                    shape :: list(),
                                    shape_signature :: list(),
                                    type :: tflite_beam_tensor_type(),
                                    quantization_params ::
                                        #tflite_beam_quantization_params{scale :: term(),
                                                                         zero_point :: term(),
                                                                         quantized_dimension :: term()},
                                    sparsity_params :: term(),
                                    ref :: reference()} |
                {error, binary()}.

Get any tensor in the graph by its id

Note that the tensor_index here means the id of a tensor. For example, if inputs/1 returns [42, 314], then 42 should be passed here to get tensor 42.

The handle borrows the interpreter's memory and keeps the interpreter alive for as long as the handle itself is reachable, so there is nothing a caller has to hold on its behalf. What the handle cannot survive is the memory moving: allocate_tensors/1, either resize_input_tensor and a second tflite_beam_interpreter_builder:build/2 all relocate what it points at, and reading through it afterwards returns {error, Reason}. Fetch it again after any of those.

-spec tensors_size(reference()) -> non_neg_integer() | {error, binary()}.

Return the number of tensors in the model.

-spec variables(reference()) -> {ok, [non_neg_integer()]} | {error, binary()}.

Get the list of variable tensors.