Skip to content

experiment: opening-pause trim × photometric augmentation (12 ACT runs) - #2

Open
parvpatodia wants to merge 1 commit into
mainfrom
exp/act-trim-augment-parv
Open

experiment: opening-pause trim × photometric augmentation (12 ACT runs)#2
parvpatodia wants to merge 1 commit into
mainfrom
exp/act-trim-augment-parv

Conversation

@parvpatodia

Copy link
Copy Markdown
Collaborator

Adds Parv's experiment record to experiments/, plus the standalone audit script it depends on.

Sibling to 2026-08-04_act-8bin-6run. This is the top+front arm — the only one of the three owners' grids without the physical wrist camera.

Two axes the 6-run grid left out

1. Opening-pause trim. The 6-run eval protocol predicted "expect a hesitant start at block-A positions… the fix is trimming dead time at collection." This tests it without re-collecting.

  • 8,722 of 72,518 frames (12.0%) are pre-teleop dead air. Left episodes average 3.53 s, right 1.38 s.
  • A 20× change in the departure threshold (0.5° → 10°) moves the cut by 1.5 percentage points — evidence it lands on genuine dead air, not slow task motion.
  • Interior pauses (4 of 119 episodes) and trailing pauses (117 of 119) are measured and deliberately left alone.
  • Nothing on disk or on the Hub is modified. The trim is a SubsetRandomSampler index list, published as Parv-09/phi_so101_8bin_v1_trim.

2. Photometric augmentation. Helps weakly and consistently (gap narrowed 6/6, ratio 6/6, val 4/6) but at matched steps raises train L1 only 0.5–6%, so it is too weak to really regularise — for a 49% throughput cost.

First robot result

The arm no longer freezes at start. ❌ It still cannot localise the duck. Per-bin success rates pending; the untrimmed control has not been run yet.

Independently reproduces the 6-run finding

Posterior collapse: raw KL 0.0969 at epoch 1 (81% of total loss) → 3e-5 by epoch 31, against the 6-run probe's kld_loss = 6.7e-5. Two arms, same conclusion.

⚠️ Reviewer notes

  • Losses in this file are NOT comparable with the 6-run file. This grid carves a 12-episode in-distribution val set out of the training pool (77 train, not 89). The held-out 30 are identical and untouched in both.
  • Everything is n=1 seed. No claim here should go in a paper or the Research Day poster as-is.
  • The file also records four corrections to earlier claims I made in this project, and an scp truncation trap that left a 163 MB stub of a 206 MB checkpoint while exiting 0.

Scope

src/phi/utils/deadtime.py is included because the experiment links it and it is standalone (verified running, with a guard against partially-fetched caches). The trainer and gap_eval.py are not here — the eval script imports split and trim helpers from the trainer, which needs packaging work. Follow-up PR.

Two axes the 6-run grid left out, on the top+front camera set (the only arm
without the physical wrist camera).

Trim: each episode's pre-teleop dead air excluded as a chunk start point.
8,722 of 72,518 frames (12.0%); left episodes average 3.53 s, right 1.38 s.
A 20x change in the departure threshold moves the cut by 1.5 points, so it
lands on genuine dead air. Interior and trailing pauses measured and left
alone. Nothing on disk or on the Hub is modified; the cut is a sampler index
list, published as Parv-09/phi_so101_8bin_v1_trim.

First rollout: the arm no longer freezes at start, which is what the trim was
for. It still cannot localise the duck. Zero per-bin numbers so far.

Also records four corrections to earlier claims made in this project, the
scp-truncation trap, and the per-variant data argument (15 episodes per bin
here vs 25 for Sherry Chen's 75% result).

Split note: this grid carves a 12-episode val set out of the training pool,
so losses are NOT comparable with the 6-run file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant