Dsxir.Optimizer.SIMBA.Sampler (dsxir v0.5.0)

Copy Markdown

Checkpointable pool state for a SIMBA session. Carries the global program pool with per-program mini-batch score lists, the shuffled trainset cursor, winning programs per step, and a threaded :rand state.

serialize/1 round-trips via :erlang.term_to_binary/2 with [:deterministic]; deserialize/2 uses [:safe] and confirms shape.

Helpers that consume randomness take and return rng_state so callers thread it explicitly; nothing mutates global :rand, which keeps checkpoint resume deterministic.

attempts counts every step/6 invocation so the budget terminates cleanly even if every candidate failed.

Summary

Types

pool_entry()
t()

Functions

build_stats(s)

Projects sampler state into a Stats record. Finalize-only fields (candidate_programs, wall_clock_ms) stay at their struct defaults.

calc_average_score(s, idx)

Mean of the score list for idx, 0.0 when no scores are recorded.

deserialize(blob, arg2)

Decodes a sampler blob produced by serialize/1. Uses the :safe term decoder and validates the resulting struct shape.

poisson(lambda, rng_state)

Knuth's Poisson sampler over rng_state. Deterministic given the state; returns the draw and the updated state.

register_new_program(s, program, score_list)

Appends program to the pool under a fresh monotonic index, records its mini-batch score_list, and advances next_idx.

serialize(s)

Encodes the sampler deterministically. Returns {:ok, blob, version}.

softmax_sample(s, idxs, temperature, rng_state)

Weighted pool-index choice with weights exp(score / temperature). Falls back to a uniform pick when the weight sum is non-positive. Deterministic given rng_state; returns the updated state.

top_k_plus_baseline(s, k)

Pool indices sorted by average score descending, truncated to k, with the baseline (index 0) always folded in.

Types

pool_entry()

@type pool_entry() :: %{idx: non_neg_integer(), program: Dsxir.Program.t()}

t()

@type t() :: %Dsxir.Optimizer.SIMBA.Sampler{
  attempts: non_neg_integer(),
  best_so_far: {non_neg_integer(), float()} | nil,
  config: map(),
  data_indices: [non_neg_integer()],
  degraded: boolean(),
  instance_idx: non_neg_integer(),
  next_idx: non_neg_integer(),
  program_scores: %{required(non_neg_integer()) => [float()]},
  programs: [pool_entry()],
  rng_seed: integer(),
  rng_state: term(),
  seed_program: Dsxir.Program.t(),
  total_planned_trials: pos_integer(),
  trainset: [Dsxir.Example.t()],
  trial_logs: %{optional(non_neg_integer()) => map()},
  winning_programs: [Dsxir.Program.t()]
}

Functions

build_stats(s)

@spec build_stats(t()) :: Dsxir.Optimizer.SIMBA.Stats.t()

Projects sampler state into a Stats record. Finalize-only fields (candidate_programs, wall_clock_ms) stay at their struct defaults.

calc_average_score(s, idx)

@spec calc_average_score(t(), non_neg_integer()) :: float()

Mean of the score list for idx, 0.0 when no scores are recorded.

deserialize(blob, arg2)

@spec deserialize(binary(), pos_integer()) ::
  {:ok, t()}
  | {:error, :version_mismatch | :corrupt_blob | {:bad_sampler_shape, term()}}

Decodes a sampler blob produced by serialize/1. Uses the :safe term decoder and validates the resulting struct shape.

poisson(lambda, rng_state)

@spec poisson(float(), term()) :: {non_neg_integer(), term()}

Knuth's Poisson sampler over rng_state. Deterministic given the state; returns the draw and the updated state.

register_new_program(s, program, score_list)

@spec register_new_program(t(), Dsxir.Program.t(), [float()]) :: t()

Appends program to the pool under a fresh monotonic index, records its mini-batch score_list, and advances next_idx.

serialize(s)

@spec serialize(t()) :: {:ok, binary(), 1}

Encodes the sampler deterministically. Returns {:ok, blob, version}.

softmax_sample(s, idxs, temperature, rng_state)

@spec softmax_sample(t(), [non_neg_integer()], float(), term()) ::
  {non_neg_integer(), term()}

Weighted pool-index choice with weights exp(score / temperature). Falls back to a uniform pick when the weight sum is non-positive. Deterministic given rng_state; returns the updated state.

top_k_plus_baseline(s, k)

@spec top_k_plus_baseline(t(), pos_integer()) :: [non_neg_integer()]

Pool indices sorted by average score descending, truncated to k, with the baseline (index 0) always folded in.

Mirrors DSPy: when 0 is absent from a non-empty top-k it replaces the last element, then the result is deduped preserving order.