View Source tflite_beam_interpreter_server (tflite_beam v1.0.0-rc2)
An interpreter that lives inside a process, so that feeding it, running it and reading the result back is one step that nothing can interleave with.
The lower-level API is not wrong -- it mirrors TfLite's C API faithfully -- but it does not say anywhere that input_tensor/3, invoke/1 and output_tensor/2 have to be treated as one operation. Two processes taking turns badly get each other's answers: measured on a real model, 147 wrong results in 400 calls, silently. This module is the answer to that, and the direct API stays exactly as it is for callers who would rather serialise access themselves.
The interpreter is handed to this process with tflite_beam_interpreter:controlling_process/2, so it cannot be reached from anywhere else even by a caller holding the reference.
Summary
Functions
Feed, run and read back, as one operation.
Feed, run and read back, with a call timeout.
Start an interpreter process outside a supervision tree.
Start an interpreter process outside a supervision tree.
Start an interpreter process for a model file.
Start an interpreter process for a model file.
Stop the process, and with it the interpreter.
Run a function against the interpreter inside the owning process.
Run a function against the interpreter inside the owning process.
Functions
Feed, run and read back, as one operation.
Concurrent callers are serialised by the process rather than racing inside the interpreter, so each gets the answer to its own input.
Feed, run and read back, with a call timeout.
The timeout gives up on the answer; it does not stop the work. When it runs out this exits the calling process, which is gen_server:call/3 behaviour and not in the return type above, and the server carries on with the inference it was given. Anything queued behind it still waits for it to finish. Raise the timeout rather than retry: a retry joins the queue behind the call it replaced.
Start an interpreter process outside a supervision tree.
Start an interpreter process outside a supervision tree.
Start an interpreter process for a model file.
Start an interpreter process for a model file.
Keyword Parameters
-spec stop(pid()) -> ok.
Stop the process, and with it the interpreter.
Run a function against the interpreter inside the owning process.
For the sequences predict/2 does not cover -- resizing an input and reallocating, say, or driving a signature runner. The function runs in this process, so nothing else touches the interpreter while it does, and it should return promptly for the same reason.
Run a function against the interpreter inside the owning process.
Same as predict/3 on the timeout: it ends the wait, not the work. A callback that outlives its timeout keeps the interpreter to itself until it returns.