MLServe.Backend.Function (MLServe v0.1.0)

Copy Markdown View Source

Serves an ordinary Elixir function as a model.

The simplest backend there is, and more useful than it looks. Rules engines, heuristics, score thresholds and glue around a remote service are all just functions, and wrapping one in MLServe gives it the same pooling, batching, caching, telemetry and versioning as a neural network — with no ML runtime involved.

It is also the backend the test suite runs on, which is deliberate: MLServe's own tests must not need a multi-gigabyte model or a Rust toolchain.

Configuration

config :ml_serve,
  models: [
    risk_score: [
      backend: MLServe.Backend.Function,
      config: [
        predict: fn %{amount: amount} -> {:ok, %{risky?: amount > 10_000}} end
      ]
    ]
  ]

Options

  • :predict — required. A 1-arity function, or {module, function, extra_args} receiving the input as the first argument. Returning a bare value is fine; it is wrapped in {:ok, value}.
  • :batch_predict — optional. A 1-arity function over the list of inputs, returning a list of results. Supply it to make MLServe.batch_predict/3 a single call.
  • :init — optional. A 0-arity function run once at load, whose result is passed to :predict as {state, input} instead of a bare input. Use it to build a lookup table.

Concurrency

Declares concurrency: :shared, so predictions run in the calling process and no worker processes are started. A plain function has no session to serialise access to, and routing it through a pool would add message copies and a bottleneck for nothing.

Examples

iex> {:ok, state} = MLServe.Backend.Function.load(predict: &(&1 * 2))
iex> MLServe.Backend.Function.predict(state, 21)
{:ok, 42}