pb_sim (faber_tweann v2.4.0)

View Source

Cart-pole balancing scape: single and double pole, with or without velocity.

Faithful port of Gene Sher's DXNN2 pole-balancing scape (Handbook Ch 14), adapted to faber-tweann's scape behaviour (one process, State threaded by the generic loop, no process dictionary).

One physics engine drives four problem variants, selected by the SENSOR's parameters (the variant tag) and the ACTUATOR's parameters (single vs double, goal length):

sensor [4] -> [CPos, CVel, PAngle1, PVel1] single pole, WITH velocity sensor [2] -> [CPos, PAngle1] single pole, NO velocity sensor [6] -> [CPos, CVel, PAngle1, PAngle2, PVel1, PVel2] double pole, WITH velocity sensor [3] -> [CPos, PAngle1, PAngle2] double pole, NO velocity

The no-velocity variants are only Markov-solvable with recurrence (the network must integrate velocity itself), so they need working recurrent evaluation. The with-velocity variants are feedforward-solvable: full state is observed.

Actuator parameters are [Damping_Flag, DPB_Flag | GoalSteps]: DPB_Flag = 0 disables the second-pole fall check (single pole) DPB_Flag = 1 enables it (double pole) Damping_Flag = without_damping -> +1 fitness per surviving step (pure timesteps-balanced, the Table 14.1 metric); with_damping -> the shaped reward that also rewards keeping the cart and pole near centre.

The integrator (sm_double_pole/3) uses semi-implicit Euler at dt=0.01s, two sub-steps per action, force applied on the first sub-step only. Even single- pole variants integrate pole 2: its mass (0.01) barely perturbs the cart and its fall check is disabled, exactly as in DXNN2.

Summary

Functions

act(ActuatorName, Params, Output, S)

init(Overrides)

sense(SensorName, _, S)