Skip to content

tools: measure the streamed RPC — planned motion through trajex, online replanning, continuous loop - #51

Draft
Nick Hehr (HipsterBrown) wants to merge 46 commits into
mainfrom
streamed-tools
Draft

Nick Hehr (HipsterBrown) wants to merge 46 commits into
mainfrom
streamed-tools

Conversation

@HipsterBrown

@HipsterBrown Nick Hehr (HipsterBrown) commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Stacked on #48; draft. Needs the system nlopt library for online_stream (build-tagged; CI never links it). Review after #48.

What this adds

Four Go programs under tools/ (outside the module binary) plus a shared untagged package, all built to answer one question on hardware: does time-scheduled streaming track a planned path better than the paced MoveThroughJointPositions, and can the pipeline keep the stream fed while replanning?

  • tools/stream_trajectory: replays an arm-recorder session (Time = i / frequency_hz, optional linear densify) through the streamed RPC; the first thing that exercised it on hardware.
  • tools/internal/streamx (untagged, CI-tested, 24 tests): trajex tensor names/marshalling (literals, no cgo import), a joint-space path-deviation metric, goal/obstacle parsing, and the live-stream splice math (At, Window, Splice) and event helpers the online tool uses.
  • tools/plan_stream: for each -goal, plan with the machine's motion service (same goal cone as MoveToPosition), time-parameterise the waypoints with the trajex ML model service on the machine (viam:trajex:mlmodel), stream via MoveThroughJointPositionsStreamed, return to start, execute the same plan through motion's paced execute. Both runs sampled and scored against the planned polyline. -obstacle boxes give the planner something to route around.
  • tools/online_stream (//go:build nlopt): plans in-process with armplanning from the configuration the arm will be in at a stitch time, so a mid-motion event (-switch-after, repeatable -obstacle-after) is replanned and spliced into the live stream without stopping; a Stop → replan → paced-move baseline runs the same scenario. -loop cycles 2–4 poses until Ctrl-C, each leg planned while the previous executes.
  • CLAUDE.md gotchas: the nlopt tag; RobotClient.CurrentInputs fails against this module (the gripper's is ErrUnsupported), so start inputs are built with NewZeroInputs + the arm.

What it found

scenario (30 deg/s unless noted) streamed paced
straight line, 5 goals × 2 speeds dev 0.7–1.8° mean; wall = trajex duration same dev at 80 deg/s; 1.1 s floor on short moves (WaitForServosToStop's 1000 ms minimum timeout on an elbow parked 1.5° short)
goal 4 (all joints, different travel) 0.85° mean 4.1° mean / 7.5° max: short-travel joints clamped up to the 12 deg/s speed floor arrive early
obstacle detour, 8–11 waypoints, 80 deg/s 1.3–1.5° mean, 2.9° max 12.7° mean / 38° max: the arm flew through part of the planned detour
online: obstacle appears @0.5 s + goal switch @0.6 s two splices, both stitch late 0, replans 13 / 17 ms, wall 2.46 s Stop, 952 ms replan from the stopped pose, 31° max deviation on plan B, wall 3.13 s
-loop, 4 poses, 40 legs / ~70 s every leg planned in 6–26 ms, stitch late 0 throughout, dev 0.7–1.9° mean, identical numbers each lap

The replan-latency gap (13 ms vs 952 ms) is the point: planning from the predicted stitch pose, already past the obstacle, is a straight line; planning from the stopped pose next to it is a search. The paced path's 38° excursion through a planned detour is a safety finding about main and is filed separately.

Caveats: the streamed stitch late and the sampler are on the tool's clock, which leads the arm by the RPC + gate offset; trajex's ML model Infer is rest-to-rest, so every splice decelerates to rest and re-accelerates (that hitch is the measured cost of a stateless TOTG service); the sampler shares the bus with the 100 Hz stream (-sample-hz is the knob).

Prerequisites

Arm with a frame; builtin motion service; viam-server ≥ 1.1.0 with the module from #48; the trajex module with one ML model service ({"name":"trajex","api":"rdk:service:mlmodel","model":"viam:trajex:mlmodel"}); for online_stream, brew install nlopt / apt install libnlopt-dev and go run -tags nlopt ./tools/online_stream ....

Testing

go test ./... (30 tool tests), go test -tags nlopt -race ./tools/online_stream/ (10: 3 planner against the module's own kinematics, 7 producer over a fake arm), go vet tagged and untagged, gofmt -s. Every hardware-facing risk called out in review (motion plan request encoding incl. the goal cloud, sampler vs move lock, the trajex C++ service's required tensors and int64 sampling rate, gRPC Canceled on Ctrl-C) was either verified against rdk source or fixed after a hardware run.

🤖 Generated with Claude Code

https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW

…ount writes not packets, sim seams)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qv1ZbWg7v4RH7HcR6xUgAi
rdk >= v1.1.0 adds MoveThroughJointPositionsStreamed to arm.Arm; both arm
models stub it with errors.ErrUnsupported until the real implementation lands.

rdk v1.6.0's Frame.Transform no longer truncates at an out-of-bounds joint,
so oob_test.go drops the assertion that pinned the old bug. ComputeOOBPosition
still clamps to the limit; whether it should stay at all is a follow-up.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qv1ZbWg7v4RH7HcR6xUgAi
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qv1ZbWg7v4RH7HcR6xUgAi
…deg start gate

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qv1ZbWg7v4RH7HcR6xUgAi
Name the ungated mid-stream ceiling at the code site and pin it with a test;
make the Stop test deterministic; Clock.WaitUntil measures against its own Now.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qv1ZbWg7v4RH7HcR6xUgAi
…amed RPC

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qv1ZbWg7v4RH7HcR6xUgAi
… the client on error

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qv1ZbWg7v4RH7HcR6xUgAi
…e tool binary

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qv1ZbWg7v4RH7HcR6xUgAi
…ools/ layout

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qv1ZbWg7v4RH7HcR6xUgAi
Also adapts the streamed path to main's three-value clampPositions after the
rebase.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
…y >= 1ms

Point 0 is due at start and the clock is read again after start is captured,
so it always counted as late by a few microseconds.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
…rvos to stop

WaitForServosToStop straight after the final goal write read Moving before it
had risen (~2ms) and returned while the arm was still travelling: hardware
runs showed a 2ms settle and a 1.2 deg base-joint error at return. Reuse the
paced path's dwell, which consults Moving only from its second poll.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
…e tools PR

CheckTrajectoryTime takes only (prev, t); the simulated Stop check lives in
startMove; docs/arm.md's streaming section is a third of its length.
tools/stream_trajectory moves to the stacked tools PR with the other tools.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
…ory points

Tensor names are the trajex ABI literals; importing its Go bindings would pull in cgo.
Promotes gorgonia.org/tensor (already an rdk dependency) to a direct require.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
…oint-space polyline

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
Deterministic frame-key ordering via slices.Sorted(maps.Keys(...)) instead of
a hand-rolled keys() helper; deg<->rad conversions via rdk's utils.DegToRad/
RadToDeg instead of local math; round trajex sample times to the nearest
nanosecond so 2.01s doesn't truncate to 2.009999999s.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
…paced execute

Plans one move through the motion service's `plan` DoCommand, time-parameterises
the waypoints with the trajex ML model service on the machine, streams the samples through
MoveThroughJointPositionsStreamed, returns to the start pose, then runs the SAME plan
through motion's paced `execute` (executeCheckStart 0.1 rad, above servo droop). Both
runs are sampled at -sample-hz and scored against the planned joint-space polyline.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
…e requires

The C++ viam:trajex:mlmodel service rejects a missing
waypoint_deduplication_tolerance_rads or path_colinearization_ratio and wants
trajectory_sampling_freq_hz as int64; the Go adapter's schema is looser.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
execute drives the arm at its configured speed, so -vel-deg only shaped the
streamed run; set the arm's speed/acceleration for the paced run and restore.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
Without an obstacle every plan is a two-waypoint straight line; a box gives
the planner something to route around, so the trajectory has corners.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
On a multi-waypoint path trajex returned its first sample at ~9.96ms; the arm
rejects a stream whose first point is not at Time 0.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
…rnal/streamx

plan_stream keeps its motion-DoCommand decoder, feeder and sampler; the pure helpers a
second tool needs move to an internal package under tools/ so they stay out of the module
binary and CI keeps testing them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
…-segment scoring

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
… start configuration

Build-tagged nlopt: armplanning's IK needs the system nlopt library via cgo, so ./... skips
the package and CI never links it. go.mod picks up go-nlopt as an indirect requirement.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
…ive stream

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
… sent; round event times

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
… a Stop/replan baseline

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
…puts gotchas

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
… RPC error; rejects an empty trajectory

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
…s named honestly

The baseline gap is measured from the event, so it is called event-to-move;
stitch lateness is on the client clock and says so.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
The producer could only replay a fixed event list. `next` lets a driver enqueue
the next event as each splice lands, which is all a continuous loop needs: it is
called under `mu` in the same critical section that clears `pending`, so `done`
(nothing sent-short, nothing pending, nothing unfired) is never momentarily true
in the gap, and it appends to both `events` and `fired` so they stay the same
length. It receives `cur` as an argument because the mutex is not reentrant.

`result()` was already safe to call mid-run; only its doc comment said otherwise.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
Each leg is planned while the previous one executes and spliced in where it
rests, via the producer's next hook: the hook schedules the replan runway+margin
before the current trajectory ends (clamped at 0, so a leg shorter than that
window simply stitches late and the arm rests at the goal meanwhile), and the
leg counter lives in the replanFunc closure alone -- replan runs without the
producer's mutex, next under it, so sharing one counter would race.

-loop rejects -switch-after / -obstacle-after and ignores -baseline (its default
is true, so it cannot be a rejection); static -obstacle boxes apply to every leg.

The trace is now a mutex'd sampler so a live run can read it. One sampler for
every mode rather than a second one for the loop: two JointPositions pollers at
-sample-hz would perturb the stream being measured. The printer polls result()
every 250ms with a cursor, so each leg prints exactly once -- a splice that lands
after its own stitch time prints on the next poll, never twice and never skipped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
The arm client reports a cancel as a codes.Canceled status, which
errors.Is(context.Canceled) misses, so the summary was skipped and the tool
exited with an error. Judge the context instead, and print the stats before
judging any error.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018G9kxn9BZtxsAen11ecEmW
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant