FilteredTracker: decouple per-frame ops from measurement CV with fine-grained locking - #66
Merged
Merged
Conversation
…-grained locking The per-frame predictStep/updateVIOCameraPose/getFilteredTransform and the heavy measurementUpdate previously had to be externally serialized (the ObjC bridge ran them on one serial queue) because the C++ object had no internal locking. That meant every per-frame call stalled for the full 0.5-2s localization wall time — AR tracking froze during each localization attempt. Make FilteredTracker internally thread-safe with a single state_mutex_ guarding the filter strategy, VIO pose state, timing and last_transform_result_. The lightweight per-frame methods lock for microseconds. measurementUpdate runs the expensive base_tracker_ localization WITHOUT the lock (snapshot the prediction up front, do CV unlocked, fuse the result under a brief lock), so it never blocks the per-frame thread. Because per-frame predictStep now keeps advancing the filter during the unlocked CV, the measurement (computed from the capture-time image) would otherwise be fused against a state that has moved on — producing a spurious innovation equal to the camera motion during CV (and likely outlier rejection). Time-align the measured pose forward by the VIO motion that elapsed during localization before fusing it: T_lar_from_camera(now) = T_lar_from_camera(capture) * vio(capture)^-1 * vio(now) Phase 3 also re-checks isInitialized() under the lock to handle a concurrent reset(). CALLER CONTRACT: base_tracker_ is not mutex-guarded, so measurementUpdate must not run concurrently with itself or with getBaseTracker() mutations (configureImageSize); those heavy/rare ops stay serialized on a single background context. Per-frame ops are always safe concurrently. Verified: builds clean, full test suite (111 tests) passes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The lightweight per-frame ops (
predictStep/updateVIOCameraPose/getFilteredTransform) and the heavymeasurementUpdatepreviously had to be externally serialized — the lar-swift ObjC bridge ran them all on a single serial dispatch queue — becauseFilteredTrackerhad no internal locking and shares mutable state (the filter strategy + VIO poses) between them.Consequence: every per-frame call stalled for the entire 0.5–2 s localization wall time. AR tracking froze during each localization attempt (raised as a code-review finding on lar-swift PR #15).
Change
Make
FilteredTrackerinternally thread-safe with a singlestate_mutex_guarding the filter strategy, VIO pose state, timing andlast_transform_result_.measurementUpdateruns the expensivebase_tracker_localization without the lock:Filter-timing correctness
Because per-frame
predictStepnow keeps advancing the filter during the unlocked CV, the measurement (from the capture-time image) would otherwise be fused against a state that has moved on — a spurious innovation equal to the camera motion during CV (→ likely outlier rejection during motion). The measured pose is time-aligned forward by the VIO motion that elapsed during localization before fusing:Phase 3 also re-checks
isInitialized()under the lock to handle a concurrentreset().Caller contract
base_tracker_is intentionally not mutex-guarded (that's the point — CV runs unlocked), someasurementUpdatemust not run concurrently with itself or withgetBaseTracker()mutations (configureImageSize). Those heavy/rare ops stay serialized on a single background context. Per-frame ops are always safe concurrently.Validation
filtered_tracker.cppcompiles with no new warnings.Follow-up (lar-swift)
Once merged + submodule bumped, the bridge (
LARFilteredTracker.mm) can drop its serial queue: per-frame ops call C++ directly (guarded bystate_mutex_), and onlymeasurementUpdate/configureImageSizestay on a background serial queue per the caller contract. This supersedes the interimdispatch_asyncmitigation.🤖 Generated with Claude Code