Overview · Quickstart · Policies & Assets · Documentation
Recorded G1 tennis policy motion. Successive poses are arranged spatially for visualization.
MimicX learns humanoid tracking policies from video-derived motion and uses policy execution feedback to refine their training supervision. Rollout diagnosis identifies difficult time windows and body regions; bounded objective and curriculum candidates guide policy continuation. Repeated execution checks determine whether to accept a candidate or retain the current policy.
Skill construction: Human video · Human reconstruction · Robot reference · Tracking policy
Policy feedback: Policy rollout · Failure diagnosis · Supervision refinement · Verified continuation
| Component | What it does |
|---|---|
| Video-to-policy pipeline | Connects GVHMR / SMPL-X reconstruction, GMR retargeting and PPO tracking in MuJoCo / MjLab. |
| Task-aware AutoRefine | Uses failure windows and body-error channels to propose targeted objectives and curricula. |
| Repeated execution gate | Checks execution improvements and tracking guards before replacing the current policy; records acceptance or protection. |
| MimicX-HLoop | Coordinates CPU/GPU/I/O dependencies with sequential, bulk-synchronous and dependency-ready executors, plus artifact and selection parity checks. |
Within each automated loop, the registered reference stays fixed while training objectives and curricula are refined. Input reference preparation is a separate stage. The method documentation describes the implemented contract; the project website contains the visual comparisons and numerical results.
git clone https://github.com/NEBULIS-Lab/MimicX.git
cd MimicX
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e '.[dev,visualization]'
python -m pytest -qThe CPU test suite includes deterministic subprocess fixtures for the complete refinement cycle. It requires neither body models nor pretrained policies. Physics training and evaluation additionally use the pinned tracking backend.
| Goal | Start here |
|---|---|
| Run a released policy | Choose a recommended policy and matching reference. |
| Learn a skill from a new video | Follow video reconstruction, retargeting and tracking. |
| Refine an existing policy | Create a manifest and run or resume AutoRefine. |
| Reproduce the paper protocols | Prepare the checksummed inputs using the reproduction inventory. |
GPU entry points use the compute allocation provided by your environment. Device IDs are process-visible indices; launchers preserve inherited GPU visibility. The core package does not install a CUDA runtime.
Pretrained policies and their companion motion references are hosted on Hugging Face, separate from the source code. Start with one task:
Tennis · Football · Dance · Kung Fu
| Release | Contents |
|---|---|
| MimicX-Policies | Recommended final policies, multi-seed baselines, warmstarts, candidate checkpoints and checkpoint provenance. |
| MimicX-Assets | Matching robot references, public configurations, execution records and reproduction metadata. |
The artifact usage guide covers selective downloads and reconstruction of the exact core input bundle. Pair each policy with its recorded reference and configuration: G1 29-DoF and historical 23-DoF policies use different model layouts.
| Guide | Scope |
|---|---|
| Installation | External dependencies, pinned backend setup and model requirements |
| Video to Policy | Reconstruction, retargeting, warmstart training and AutoRefine |
| Method | Diagnosis, proposals, continuation and acceptance rules |
| Paper Reproduction | Registered inputs, controlled comparisons and evaluation protocols |
| HLoop | Heterogeneous execution, scheduling modes and parity checks |
| Visualization | Rollout videos, screenshots and exact-state replay |
| Release Verification | Tested components and artifact checks |
Repository structure
| Directory | Purpose |
|---|---|
mimicx/refinement/ |
Diagnosis-conditioned search, verification and persistent state |
mimicx/runtime/ |
Dependency-aware execution and parity checks |
mimicx/adapters/ |
Motion-format bridge |
mimicx/evaluation/ |
Metric reduction and experiment evidence utilities |
mimicx/visualization/ |
Replay, source-object registration, camera and geometry utilities |
dependencies/backend_overlay/ |
Tracking backend modifications |
dependencies/assets/ |
Attributed third-party visualization assets |
scripts/ |
Setup, conversion, training-loop and export entry points |
mimicx/configs/paper/ |
Frozen method settings and checksummed input specification |
docs/ |
Project website and reproduction documentation |
tests/ |
Portable CPU regression tests |
Website and numerical data
The project website presents the
method, paired policy videos and numerical evidence. Measured results live
only under docs/assets/results/, separate from implementation and runtime
inputs. The static page can also be opened from docs/index.html.
See website maintenance for rebuilding and publishing it.
MimicX builds on GVHMR, GMR, SMPL-X, MuJoCo, MjLab and RSL-RL. Upstream projects, pinned dependencies and their notices are listed in dependencies.
Code is released under Apache-2.0. Included Unitree hand visualization assets retain BSD-3-Clause, and the tracking backend retains its upstream notices. Body models, source videos, datasets and third-party weights follow their providers' terms; obtain the required authorized inputs separately. Demonstration imagery and scene assets retain their respective rights. Policy checkpoints are hosted on Hugging Face, not stored in Git.