Local, process-supervised TRL reinforcement backend.

The backend is intentionally narrow: it speaks the versioned Imp.Clients.TRLProtocol to the bundled worker and supports only GRPO. Constructing the backend does not install Python packages or download a model. The configured Python environment and model tree must already exist.

The package ships the worker, its locked Python environment (priv/trl_worker/pyproject.toml and uv.lock), and the default contract, which pins Qwen2.5-0.5B, the worker's dependency versions, and one durable MPS LoRA update. Build that environment outside the package, since priv is copied into _build and releases: for example UV_PROJECT_ENVIRONMENT=/path/to/trl-env uv sync --project <priv>/trl_worker, then pass /path/to/trl-env/bin/python as :python. A contract you write for longer runs is passed as :contract_path; it must pin the dependency versions the environment installs, or the worker refuses to start. Contracts accept arbitrary Imp-rendered prompt groups and finite external rewards. Ordered groups are source-bound and batched through official TRL. Later steps restore the prior adapter, optimizer, scheduler, Trainer state, and explicit MPS RNG before continuing. Experiment-specific assertions such as a required tensor change belong in an explicit contract; they are not imposed on ordinary training, where uniform-reward groups may truthfully produce no-op steps. Mismatched rollout or step budgets fail before the worker loads the model.

Public GRPO.train_kwargs may override the narrow data-only training keys learning_rate, beta, loss_type, and scale_rewards. They are validated before worker startup, sealed into the session identity, and written to a session-owned runtime contract. Arbitrary Python kwargs are never forwarded.