Skip to content

feat(motion): add fixed-scene expert trajectory generation - #591

Open
yuecideng wants to merge 3 commits into
mainfrom
feat/fixed-scene-trajectory-generation
Open

feat(motion): add fixed-scene expert trajectory generation#591
yuecideng wants to merge 3 commits into
mainfrom
feat/fixed-scene-trajectory-generation

Conversation

@yuecideng

Copy link
Copy Markdown
Contributor

Description

Add a fixed-scene expert trajectory generation pipeline that restores the same physical initial state between attempts, augments explicitly permitted motion phases, validates the executed motion, and counts an episode only after its LeRobot shard has been sealed and read back.

The parallel cube example keeps the cube's initial pose fixed while varying 0°/90° grasp orientations and transit paths. Collection checks full-joint collision geometry, native contacts at every physics substep, bilateral finger contact, lift height, and held-object drift before persisting an episode.

  • Group solvers, planners, workspace, and augmentation under the lazily loaded embodichain.lab.sim.motion package. Update imports, configuration references, API docs, examples, benchmarks, tests, and agent context.
  • Add immutable trajectory contracts, protected-phase residual/retiming operators, coverage bookkeeping, bounded generation sessions, and confirmed commit receipts.
  • Add sim/Gym initial-state ownership and restoration, candidate planning, measured rollout evidence, synchronous LeRobot persistence, and initial candidate hooks for handwritten/atomic sources.
  • Add offline MoveEndEffector → PickUp template export, a bounded CPU-physics PickUp validator, repeated parallel collection, and synchronized H.264 video examples.

Breaking API migration: replace embodichain.lab.sim.{solvers,planners,workspace} imports with embodichain.lab.sim.motion.{solvers,planners,workspace}. Legacy import packages are removed; low-level Warp kernels remain under utils/warp/kinematics.

Dependencies: the optional trajectory-generation extra adds python-fcl, trimesh, and yourdfpy for PickUp collision validation. Collection uses the existing DexSim, LeRobot, and video dependencies; the free-motion example additionally uses cuRobo.

This draft contains the implemented portions of the design and implementation plan under docs/design/fixed_scene_expert_trajectory_augmentation_*. Full Atomic Runtime candidate consumption/tracking/recovery, Gym PickUp collection, the unified configuration launcher, and asynchronous persistence remain follow-up work. The current PickUp integration supports one unscaled fixed-base URDF robot and cuboid rigid objects with CPU physics; sampled geometry checks do not guarantee continuous collision avoidance.

Type of change

  • New feature
  • Breaking change (motion import paths)
  • Documentation update

Validation

  • After rebasing onto current main (f4ad72de): 1542 passed, 1 skipped, 168 deselected across motion, atomic actions, trajectory generation, Gym preparation/demo and simulation integrations, SimulationManager, articulation/gizmo, scene semantics, and motion benchmarks (excluding GPU/slow/requires_sim tests).
  • 3 passed, 1 deselected for the real parallel PickUp collection, failure cleanup, and existing visualization example:
    pytest tests/lab/trajectory_generation/test_pickup_collection.py \
      tests/lab/trajectory_generation/test_cube_grasp_parallel.py --run-gpu -m gpu -q
  • Real collection: 8 proposed / 8 attempted / 8 committed, four physical rows across two restored batches; 1.5 s hold with 100% bilateral contact coverage, minimum lift 17.90–17.92 cm, maximum relative translation drift 0.35–0.73 mm. Tests read back every LeRobot shard and the 544-frame, 20 fps, 1280×1056 H.264 video.
  • black . and git diff --check passed. Public API coverage: 1761/1761; checker tests: 8 passed. Sphinx dummy build completed with documentation warnings.
  • All 43 new Python files pass header/future-import/export checks; moved legacy modules retain their existing style. Changed canonical skills pass quick_validate.py.
  • The full repository suite and complete M1 source/host acceptance matrix were not run.

Screenshots

Generate the synchronized four-view video and verified dataset locally:

pip install -e '.[trajectory-generation]'
python examples/sim/motion/trajectory_generation/cube_pickup_collection.py \
  --output /tmp/cube-pickup-collection --episodes 8 --record-video

Outputs include preview.mp4, generation_report.json, pickup_report.json, manifest.json, and individual LeRobot episode shards. The video includes all attempted rollouts; the manifest includes only confirmed accepted episodes.

Checklist

  • I have run the black . command to format the code base.
  • I have made corresponding changes to the documentation.
  • Public API changes are reflected in the API docs (python docs/scripts/check_api_docs.py).
  • I have added tests that prove my fix is effective or that my feature works.
  • Dependencies have been updated.

Group solvers, planners, workspace and augmentation under sim.motion. Add fixed-scene preparation, bounded generation and confirmed LeRobot persistence, plus physically validated parallel cube PickUp collection and video examples.

BREAKING CHANGE: import robot motion APIs from embodichain.lab.sim.motion; previous sim.solvers, sim.planners and sim.workspace paths are removed.
@yuecideng yuecideng added docs Improvements or additions to documentation enhancement New feature or request motion gen Things related to motion generation for robot refactor breaking dataset labels Sep 5, 2026
@yuecideng
yuecideng marked this pull request as ready for review September 6, 2026 17:00
@greptile-apps

greptile-apps Bot commented Sep 6, 2026

Copy link
Copy Markdown

Greptile Summary

The PR adds a fixed-scene trajectory-generation pipeline with deterministic state restoration, bounded candidate execution, physical validation, and confirmed episode persistence.

  • Moves motion solvers, planners, workspace utilities, and augmentation APIs beneath the lazily loaded embodichain.lab.sim.motion package.
  • Adds fixed-scene host ownership, Gym preparation leases, rollout execution, collision/contact validation, and synchronous LeRobot sinks.
  • Adds offline PickUp template export, parallel collection examples, documentation, and focused integration tests.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains within the eligible follow-up-review scope.

No blocking failure remains.

Important Files Changed

Filename Overview
embodichain/lab/trajectory_generation/runner.py Coordinates bounded candidate generation, restoration, measured rollout validation, and receipt-driven commit accounting.
embodichain/lab/trajectory_generation/initial_state.py Introduces fixed-scene ownership, state capture and restoration, epoch checks, and prepared-batch publication.
embodichain/lab/trajectory_generation/execution.py Adds qpos rollout execution through Gym or direct simulation while collecting measured observations and validation evidence.
embodichain/lab/trajectory_generation/integrations/contact.py Implements PickUp collision, native-contact, bilateral-grasp, lift, hold-duration, and drift qualification.
embodichain/lab/trajectory_generation/sinks.py Adds synchronous LeRobot shard persistence with sealing, readback verification, manifests, and confirmed commit receipts.
embodichain/lab/gym/envs/embodied_env.py Adds controlled full-batch generation preparation that clears prior episode state and enables stepping only after successful verification.
embodichain/lab/sim/atomic_actions/primitives/pick_up.py Extends ragged grasp handling with explicit candidate masks and safe treatment of empty or failed IK rows.
embodichain/lab/sim/motion/planners/curobo/curobo_planner.py Updates cuRobo planning integration and cache behavior for the relocated motion package and fixed-scene workflow.

Sequence Diagram

sequenceDiagram
    participant R as GenerationRunner
    participant H as FixedSceneHost
    participant P as Candidate Planner
    participant E as Rollout Executor
    participant V as Motion Validator
    participant S as LeRobot Sink
    R->>H: Acquire case and capture initial state
    loop Candidate batches
        R->>H: Restore and verify initial state
        R->>P: Plan bounded candidates
        P-->>R: Valid candidate trajectories
        R->>E: Execute selected trajectory
        E->>V: Report measured states and substep contacts
        V-->>R: Acceptance evidence
        alt Accepted
            R->>S: Submit episode
            S->>S: Seal and read back shard
            S-->>R: Confirmed commit receipt
        else Rejected
            R->>R: Record failed attempt
        end
    end
    R->>H: Release ownership and close
Loading

Reviews (2): Last reviewed commit: "fix(examples): configure trajectory visu..." | Re-trigger Greptile

@yuecideng
yuecideng requested a review from matafela September 7, 2026 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking dataset docs Improvements or additions to documentation enhancement New feature or request motion gen Things related to motion generation for robot refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant