Hi, thanks for ASAP.
I benchmarked 29 open-source Unitree G1 walking policies behind one C++ interface, on one task: 60 seconds, 12 waypoints, 50 Hz control over a 2 ms step, a crane releasing after a 3 s shared stance, and a punch on every segment (random joint, random direction, 0.08 s, up to 600 N). The policy owns the 15 leg and waist joints; the 14 arm joints random-walk under the harness.
asap came 11th of 29: 12 % MuJoCo / 16 % PhysX, 123 cm / 34° error.
I want to be upfront that I think this number says more about the fit between ASAP and my task than about ASAP. The checkpoint I use is model_6600.onnx, and my reading is that it is a motion-tracking policy, whereas my tour asks for open-ended waypoint following under disturbance with no reference motion. If that is a category error on my part, I would rather say so in the table than leave a misleading 12 %.
For completeness, how I drive it: 12 leg actions, a 100-dim frame with 4 frames of history (500-dim input), ACTION_SCALE = 0.25, gait period 0.9 s, base height command 0.78 m scaled by 2.0, ang-vel scale 0.25, dof-vel scale 0.05. The frame is laid out as actions(12), ang-vel(3), cmd yaw, cmd height, cmd lin(2), cmd stand, cos phase, dof pos... I enter a walking regime at 0.08 m / 0.10 rad from target and exit at 0.04 m / 0.05 rad. The 17 upper joints from index 12 are fed from the harness's random-walking arms.
What would help most:
- Is there a locomotion-oriented ASAP checkpoint I should be using instead of
model_6600?
- If not, is it fair to mark ASAP in my table as evaluated outside its intended regime? I would rather annotate it accurately than rank it.
- The stand/walk hysteresis and the fixed 0.9 s gait period are mine and may be fighting the policy.
Happy to take a PR against policies/asap/policy.cpp, or to add an annotation if the honest answer is that this task is not what ASAP is for.
https://github.com/rhoyn/teleop-walking-benchmark
https://rhoyn.com/stable-walk
Hi, thanks for ASAP.
I benchmarked 29 open-source Unitree G1 walking policies behind one C++ interface, on one task: 60 seconds, 12 waypoints, 50 Hz control over a 2 ms step, a crane releasing after a 3 s shared stance, and a punch on every segment (random joint, random direction, 0.08 s, up to 600 N). The policy owns the 15 leg and waist joints; the 14 arm joints random-walk under the harness.
asapcame 11th of 29: 12 % MuJoCo / 16 % PhysX, 123 cm / 34° error.I want to be upfront that I think this number says more about the fit between ASAP and my task than about ASAP. The checkpoint I use is
model_6600.onnx, and my reading is that it is a motion-tracking policy, whereas my tour asks for open-ended waypoint following under disturbance with no reference motion. If that is a category error on my part, I would rather say so in the table than leave a misleading 12 %.For completeness, how I drive it: 12 leg actions, a 100-dim frame with 4 frames of history (500-dim input),
ACTION_SCALE = 0.25, gait period 0.9 s, base height command 0.78 m scaled by 2.0, ang-vel scale 0.25, dof-vel scale 0.05. The frame is laid out as actions(12), ang-vel(3), cmd yaw, cmd height, cmd lin(2), cmd stand, cos phase, dof pos... I enter a walking regime at 0.08 m / 0.10 rad from target and exit at 0.04 m / 0.05 rad. The 17 upper joints from index 12 are fed from the harness's random-walking arms.What would help most:
model_6600?Happy to take a PR against
policies/asap/policy.cpp, or to add an annotation if the honest answer is that this task is not what ASAP is for.https://github.com/rhoyn/teleop-walking-benchmark
https://rhoyn.com/stable-walk