Dsxir.Optimizer.Search.TPE (dsxir v0.4.0)

Copy Markdown

Categorical Tree-structured Parzen Estimator. Implementation of Dsxir.Optimizer.Search.Sampler.

No tree structure (all dims unconditional). No continuous dims. Pure Elixir, no Nx.

Hyperparameters (settable via init/2 opts):

  • :gamma (default 0.20) — quantile splitting good vs. bad observations.
  • :alpha (default 1.0) — Laplace smoothing constant.
  • :n_ei_candidates (default 24) — candidates sampled from l before EI scoring.
  • :cold_start_trials (default 10) — uses Random until this many observations.
  • :seed (default 0) — PRNG seed.

Summary

Types

t()

Functions

init(space, opts \\ [])

Builds a TPE sampler state for the given categorical space.

observe(state, observations)

No-op. TPE rebuilds its good/bad partition from the full history on each suggest/3, so there is no rolling state to update.

suggest(state, history, n)

Suggests n candidate configs.

Types

t()

@type t() :: %Dsxir.Optimizer.Search.TPE{
  alpha: float(),
  cold_start_trials: non_neg_integer(),
  gamma: float(),
  n_ei_candidates: pos_integer(),
  rand_state: :rand.state(),
  random_delegate: Dsxir.Optimizer.Search.Random.t(),
  space: Dsxir.Optimizer.Search.Sampler.space()
}

Functions

init(space, opts \\ [])

@spec init(
  Dsxir.Optimizer.Search.Sampler.space(),
  keyword()
) :: t()

Builds a TPE sampler state for the given categorical space.

See module docs for supported options.

observe(state, observations)

@spec observe(t(), [Dsxir.Optimizer.Search.Sampler.observation()]) :: t()

No-op. TPE rebuilds its good/bad partition from the full history on each suggest/3, so there is no rolling state to update.

suggest(state, history, n)

@spec suggest(t(), [Dsxir.Optimizer.Search.Sampler.observation()], pos_integer()) ::
  {[Dsxir.Optimizer.Search.Sampler.config()], t()}

Suggests n candidate configs.

While history has fewer than :cold_start_trials entries, delegates to Dsxir.Optimizer.Search.Random. Once engaged, splits history by :gamma into good/bad sets, samples :n_ei_candidates from the Laplace-smoothed categorical l, scores each candidate by prod_d l_d(x_d) / g_d(x_d), and returns the top n.