MicrogradEx.Losses (MicrogradEx v0.1.0)

Copy Markdown View Source

Loss functions for small scalar MicrogradEx models.

The main Livebook demo uses the same max-margin classification objective as the official Python micrograd notebook: mean SVM-style hinge loss plus L2 regularization over model parameters.

Summary

Types

opts()

Functions

max_margin(model, xs, ys, opts \\ [])

Computes max-margin classification loss for signed labels.

Types

opts()

@type opts() :: [
  alpha: number(),
  batch_size: nil | pos_integer(),
  seed: {integer(), integer(), integer()} | nil
]

Functions

max_margin(model, xs, ys, opts \\ [])

@spec max_margin(MicrogradEx.NN.model(), [[number()]], [number()], opts()) ::
  MicrogradEx.Losses.Result.t()

Computes max-margin classification loss for signed labels.

Labels may be -1, -1.0, 1, or 1.0; they are normalized to floats internally. The returned losses are %MicrogradEx.Value{} structs so callers can run MicrogradEx.Value.backward/1 on result.total_loss.

Options

  • :alpha - non-negative L2 regularization scale, default 1.0e-4
  • :batch_size - nil for full batch or a positive integer
  • :seed - deterministic mini-batch seed, default {1337, 1337, 1337}

Example

iex> alias MicrogradEx.{Losses, NN}
iex> model = MicrogradEx.NN.MLP.new(2, [2, 1], seed: {1, 2, 3})
iex> result = Losses.max_margin(model, [[-1.0, 0.0], [1.0, 0.0]], [-1.0, 1.0])
iex> %MicrogradEx.Value{} = result.total_loss
iex> result.accuracy >= 0.0 and result.accuracy <= 1.0
true