MLServe.BackendError exception (MLServe v0.1.3)

Raised when a backend callback raises, throws, or exits.

MLServe wraps every backend invocation. When the backend fails in a way it did not report as {:error, reason}, the original kind, reason and stacktrace are captured here rather than discarded, and the whole struct is returned as {:error, {:backend_error, %MLServe.BackendError{}}}.

Keeping the stacktrace is the point: a model that raises ArgumentError deep inside a tensor operation is a bug you need the line number for, and a {:error, :something_went_wrong} atom would throw that away.

Fields

:backend — the backend module that failed
:callback — the callback that failed, e.g. {:predict, 2}
:kind — :error, :throw or :exit
:reason — the raised exception, thrown value, or exit reason
:stacktrace — the stacktrace at the point of failure
:model / :version — which model was being served

Summary

Types

t()

Types

t()

@type t() :: %MLServe.BackendError{
  __exception__: true,
  backend: module() | nil,
  callback: {atom(), arity()} | nil,
  kind: :error | :throw | :exit,
  message: String.t(),
  model: atom() | nil,
  reason: term(),
  stacktrace: Exception.stacktrace(),
  version: String.t() | nil
}