A pole is hinged on top of a cart that runs on a straight track. Left alone it falls over; your controller must push the cart back and forth to keep it upright, and keep the cart near the middle of the track. Every 0.02 seconds the simulator calls
controller(t, x, v, angle, omega) -> force
with the time t in seconds (0, 0.02, ...), the cart's position x (m, 0 is the centre, positive to the right) and speed v (m/s), and the pole's angle from upright (rad, positive when the top leans right) and its rate omega (rad/s). Return the force on the cart in newtons, positive pushing it right; anything outside -10..10 N is limited to that range.
The physics. Cart mass M = 1.0 kg, pole mass m = 0.1 kg, half the pole's length l = 0.5 m, g = 9.81. Each test changes each of M, m and l by up to 10 % (a different cart; you are not told). With s = sin(angle), c = cos(angle), the simulator computes
temp = (F + m*l*omega**2*s) / (M + m)
angle_acc = (g*s - c*temp) / (l * (4/3 - m*c*c / (M + m)))
x_acc = temp - m*l*angle_acc*c / (M + m)
and moves the state on in four steps of 0.005 s, each v += 0.005*x_acc; x += 0.005*v; omega += 0.005*angle_acc; angle += 0.005*omega, holding your force for the whole 0.02 s. The readings are noisy (about 0.002 m and 0.002 rad on the positions, 0.01 on the rates). The cart starts up to 0.5 m from the centre, the pole up to 0.1 rad from upright, both at rest. Twice in each run, once between 3 and 6 s and once between 7 and 9 s, someone nudges the pole: its rate jumps by 0.6 to 1.0 rad/s, to the left or the right.
The run lasts 10 s (500 calls). The pole has fallen if |angle| > 0.6 rad, and the cart has left the track if |x| > 2.4 m; either ends the run.
Each test calls run_balance(controller, seed) (defined for you; try it with Run), which returns a summary such as
{"fell_at": None, "rms_angle_deg": 2.3239, "rms_cart": 0.3815, "rms_force": 1.0256, "cost": 9.3012,
"max_angle_deg": 5.942, "max_cart": 0.6169}
for lambda t, x, v, angle, omega: 1 * x + 2 * v + 30 * angle + 6 * omega on seed 1. Your function may keep state in global variables; t == 0 marks the start of a new run.
How this problem is scored
- A test passes when the pole never falls and the cart never leaves the track.
- The run's cost is
rms_angle_deg ** 2 + 25 * rms_cart ** 2 + 0.25 * rms_force ** 2: the mean square of the angle in degrees, of the cart's distance from the centre (1 m counts like 5°) and of the force (2 N count like 1°), over all steps. - Its quality is
100 * ln(30 / cost) / ln(7.5), kept between 0 and 100: a cost of 30 scores 0 and a cost of 4 scores 100; each halving of the cost adds about 34 points. - The gains in the example above average about 60 over the tests. Well-chosen gains for all four states reach the mid 80s.
Constraints
- return a number; the result must not depend on the clock, and each call should take well under a millisecond
Goals
- Stabilise an unstable plant with feedback from all four of its states
- Discover that keeping the cart near the centre needs a position term with a surprising sign
- Trade uprightness and centring against force, guided by the closed-loop poles