Skip to main content
The exam is the sole scoreboard. It is frozen: same two worlds (arena, slalom), same controller tick, same proximity remap, same episode count, for every job ever run, so results are comparable across the whole program history.

​
Contract

Every job.json records the contract it was examined under:
20 episodes per world, 172 steps per episode (60 s at a 350 ms tick), fixed spawn slots.

​
The four bars (per world)

verdict.pass requires all four in both worlds. verdict.why lists each failed bar, quoting the worst episode per world.

​
Reading verdicts honestly

No policy in program history has passed all four bars. The best-known profile, reproducible via lineage jobs, is a contact-safe wanderer: 0 contacts in 40/40 episodes, but short of the clearance, coverage, and path bars. A pass: false with clean contacts is state of the art, not failure.
Judge from per-episode rows, not verdict lines: the verdict quotes minima, and fixed spawn slots collapse any wall-saturated policy into identical worst episodes, so two very different policies can produce the same why lines.

​
Anti-exploit machinery

Frozen environments get gamed, so every exam carries control arms:
  • Cruiser control: a fixed forward-driving policy runs alongside your champion and must rack up contacts (summary.cruiser_contacts = 20/world). If it doesn’t, the exam is vacuous and the run is invalid, not a pass.
  • Dream fitness (dream_F) is reported but never part of the verdict. Policies that exploit the dream get exposed by the oracle.

​
Determinism

  • Training: bit-deterministic for a given seed on a fixed platform. The same spec on macOS-arm64 vs Linux-amd64 yields different champions; the floating-point paths differ.
  • Exam: deterministic to aggregates; per-episode values drift at the third decimal across launches (cross-launch physics noise).

​
Policy format: bring your own

The exam accepts either:
  • a 9-float controller (genome9, or the genome6 it expands from), or
  • your own policy as a sandboxed Python module (policy.format = python_module); see Certify your own policy.
A python_module policy implements reset(seed) + act(obs), where obs is the same observation dict the local gym emits and the action is the wheel-fraction vocabulary, so a policy trained locally drops into the exam unchanged. The policy replaces only the champion arm; everything else here stays frozen, and every exam-stage change passes a locked take-7 acceptance replay before serving.