feat(cloud): opt-in raw uploads with per-shot IWR6843 L3 dumps - #287
Merged
Merged
Conversation
`openflight-cloud raw on` (off by default) uploads sessions unfiltered
(minus kld7_buffer), saves IWR6843 dumps even without --debug, and PUTs
each shot's dump to /v1/sessions/{id}/captures/iwr6843/{shot_number}
after the session uploads, matched by shot_number.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Raw-mode configuration, queue durability, and dry-run behavior have unresolved correctness and privacy issues.
Review effort: Balanced
Findings: 1
Open (7)
What changed in this PR
Adds opt-in raw cloud uploads, including IWR6843 L3 dump collection and per-shot upload support.
Changes:
- Adds raw-mode configuration, CLI controls, filtering, and upload APIs.
- Persists and retries per-shot capture queues.
- Adds documentation and comprehensive tests.
| File | Description |
|---|---|
tests/test_cloud_raw_uploads.py |
Tests raw filtering, uploads, retries, CLI, and status. |
src/openflight/server.py |
Enables IWR6843 dump saving in raw mode. |
src/openflight/cloud/spool.py |
Persists capture queues in pushed markers. |
src/openflight/cloud/filtering.py |
Adds raw-mode filtering and capture discovery. |
src/openflight/cloud/config.py |
Adds the upload_raw setting. |
src/openflight/cloud/commands.py |
Orchestrates capture uploads and reporting. |
src/openflight/cloud/client.py |
Adds the binary capture upload endpoint. |
src/openflight/cloud/cli.py |
Adds raw on|off. |
docs/using/cloud-sync.md |
Documents raw upload usage. |
docs/reference/cloud-uploader-spec.md |
Defines the raw upload wire contract. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| session_id, | ||
| capture.get("kind", filtering.IWR6843_CAPTURE_KIND), | ||
| capture["shot_number"], | ||
| capture_path.read_bytes(), |
Comment on lines
+237
to
+240
| has_captures = config.upload_raw and any( | ||
| spool.pending_captures(p) for p in spool.session_files(log_dir) if spool.is_pushed(p) | ||
| ) | ||
| if not pending and not (has_captures and not dry_run): |
Comment on lines
+213
to
+215
| marker["captures_pending"] = pending | ||
| marker["captures_parked"] = list(marker.get("captures_parked") or []) + newly_parked | ||
| _write_json(_sidecar(path, PUSHED_SUFFIX), marker) |
| try: | ||
| from .cloud.config import load_config | ||
|
|
||
| config = load_config() |
Comment on lines
+193
to
+195
| ## 4b. Opt-in raw uploads (radar testing) | ||
|
|
||
| A device whose owner runs `openflight-cloud raw on` (config `"upload_raw": true`) |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds an opt-in raw upload mode to the cloud uploader, so testers can send raw radar data and IWR6843 L3 dumps to FlightWeb.
openflight-cloud raw on|offsetsupload_rawin cloud.json. It is off by default, and filtered uploads are unchanged for everyone else.PUT /v1/sessions/{id}/captures/iwr6843/{shot_number}, and FlightWeb matches it to the shot by that number.The server side is jewbetcha/flightweb#1.
Why was this required?
We need raw radar data (OPS243 rolling buffers and IWR6843 L3 dumps) from real sessions to test and tune the radar pipelines. Today the uploader strips all of it, and L3 dumps never leave the Pi. The dumps are only useful when paired with each shot's measured data, so they have to be matched to their shot_detected entries. Keeping it opt-in preserves the "raw radar data never leaves your Pi" default for everyone else.
Automated tests
New
tests/test_cloud_raw_uploads.py(16 tests) covers:Full suite: 1628 passed, 10 skipped. Ruff is clean, and pylint has no new warning types.
Manual (human) testing
push --dry-runand status with raw on against a real IWR session (7 shots). Dry-run listed rolling_buffer_capture, iwr6843_capture and the other raw entries, and found all 7 dump references. The dumps showed as not on disk because the paths point at the Pi.Checklist
uv run pytest tests/ -v)uv run pylint src/openflight/ --fail-under=9)uv run ruff check src/openflight/)cd ui && npm run build) (no UI changes)cd ui && npm run lint) (no UI changes)🤖 Generated with Claude Code