Probabilistic reversal bandit: the FAIR lifetime-learning contest.

Like reversal_bandit_sim, but (a) reward is NOISY -- the good arm pays +1 with probability PHi (e.g. 0.8) and -1 otherwise, the bad arm the reverse -- so a single trial is uninformative and the agent must INTEGRATE evidence over trials; and (b) the previous trial's reward and action are fed back as SENSOR inputs, so a recurrent STATE can compete (RL^2-style meta-learning) on equal footing with reward-gated plasticity. The noise defeats a reactive win-stay-lose-shift, so this genuinely tests learning, not a one-step reflex.

Sensor (vl 3): [LastReward, LastAction, 1.0]. LastReward -1.0 / +1.0 (0.0 on the first trial) LastAction +1.0 if the last choice was arm 0, -1.0 if arm 1 (0.0 first trial) 1.0 constant bias. Actuator (vl 1): Output of zero or more chooses arm 0, below zero chooses arm 1. Reward / modulator: +1.0 / -1.0 as above (also the neuromodulator for plasticity).

Reward is drawn deterministically from a hash of (FlipSeed, Trial, Arm), so the environment's "coin flips" are FIXED per instance: two agents that make the same choice at the same trial get the same reward (common random numbers), and the draw never touches the global RNG the evolver uses. Fully determined by its init params [GoodArm, ReversalAt, Lifetime, FlipSeed, PHi].

Baselines: a fixed policy scores chance across a balanced instance set; an agent that integrates reward and adapts (in state or in weights), and re-adapts after the reversal, approaches the optimal mean reward 2*PHi - 1 per trial.