Evaluate-and-select meta-optimizer for prompt and weight optimization sequences.
Imp evaluates the original program and every successfully compiled strategy prefix. With validation data it returns the highest-scoring candidate, with earlier candidates winning ties; without validation it returns the latest successful candidate. Compilation stops at the first failed step.
When valset_ratio is positive and no validation set is supplied, Imp keeps
at least one row for validation whenever the trainset has two or more rows.
This avoids silently changing the default into no-validation/latest-prefix
selection on small datasets. A one-row trainset cannot be split while
retaining training data, so it remains entirely available for training and
uses no automatic validation row.
Training steps contribute a candidate only after returning a completed,
rebound Imp.Optimizer.TrainingResult. Imp.Clients.TrainingJob results are
polled to a terminal state under a bounded timeout and pending jobs receive a
bounded cancellation attempt; the sequence never advances on a merely
submitted job. Generic asynchronous optimizers must return this job type. The
default weight optimizer has no inferred provider: callers must replace it or
configure a BootstrapFinetune trainer before a weight-bearing strategy can run.
max_errors and num_threads, given to Imp.optimize/5 as invocation
options, control BetterTogether's
baseline and prefix-selection evaluations. Child-specific compile options go
in optimizer_compile_args; for example, COPRO's internal trainset evaluation
accepts num_threads and max_errors there. Declared child options are
validated before baseline evaluation, and no option is silently discarded.
Summary
Types
@type step_request() :: %{ optimizer: struct(), program: struct(), trainset: list(), validation: list() | nil, teacher: teacher(), invocation_opts: keyword(), training_launch_timeout: pos_integer(), training_timeout: non_neg_integer(), training_poll_interval: non_neg_integer(), training_cancellation_timeout: non_neg_integer() }