Skip to content

fix(capture-linux): measure the cursor against the window, not its mo… - #514

Draft
bosskopp wants to merge 1 commit into
getopenscreen:mainfrom
bosskopp:fix/linux-window-capture-cursor-crop
Draft

fix(capture-linux): measure the cursor against the window, not its mo…#514
bosskopp wants to merge 1 commit into
getopenscreen:mainfrom
bosskopp:fix/linux-window-capture-cursor-crop

Conversation

@bosskopp

@bosskopp bosskopp commented Aug 27, 2026

Copy link
Copy Markdown

Summary

Recording a window on Linux drew the cursor overlay in the wrong place for the whole take — a constant offset, present from the first frame, visible in a three-second clip. Full-screen capture was unaffected.

mutter never renegotiates the format for a window stream. It pins the stream to the window's monitor and carves the window out through SPA_META_VideoCrop, which can move on any buffer as the window does. The encoder already reads through that rect (Capture::read_origin), so the MP4 holds the window — but the portal reports the pointer in stream pixels, measured from the monitor's corner, and emit_sample normalised it against the negotiated format. A monitor-relative position divided by monitor dimensions, painted onto window-sized footage: wrong origin and wrong scale. For a 640×480 window at (100, 50) on a 1920×1080 monitor, a pointer at the window's centre serialised as 0.219, 0.269 instead of 0.5, 0.5.

Capture now remembers the rect it actually read — read_origin at the committed size, so it follows a window that moves and keeps the clamp that stops a shrunken window reading past the buffer — and content_rect() is the single place that answers "what does the file show". emit_sample subtracts that origin and reports that rect's dimensions, so the accumulator's x / width lands in the space frame_geometry.rs already assumes.

visible moves with it: it was tested against the monitor, so a pointer that had left the recorded window still reported visible: true.

Full-screen capture is byte-identical — no crop means the content rect is the stream. A cursor-only session opens no encoder and keeps normalising against the stream, which is correct there: its video comes from Electron, not from the helper.

Files touched

File Change
electron/native/pipewire-capture/src/capture.rs Capture::content tracked in stage(); content_rect() accessor
electron/native/pipewire-capture/src/main.rs content_rect() helper; emit_sample measures against the content rect
electron/native-bridge/cursor/recording/pipeWireCursorAccumulator.ts comment only — cx = x / width was already right
technical-documentation/architecture/recording.md documents the crop/cursor coordinate contract

Related issue

Fixes #513

Type of change

  • Bug fix
  • Feature
  • Enhancement
  • Documentation
  • Refactor / maintenance
  • Performance
  • Security

Release impact

  • Patch
  • Minor
  • Major / breaking change
  • No release note needed

Desktop impact

  • Windows
  • macOS
  • Linux
  • Installer / packaging
  • Not platform-specific

Screenshots / video

Not included — the defect is a cursor-position offset in a screen recording, and a still frame cannot show where the pointer actually was. The numeric case in the summary (0.219, 0.269 vs 0.5, 0.5) is what the tests pin instead.

Testing

cargo test in electron/native/pipewire-capture69 pass (63 before, 6 new). Note that CI does not build or test this crate (the only cargo job in ci.yml is the macOS compositor), so this is the only automated signal on it.

New tests:

  • capture::tests::the_content_rect_is_the_window_the_file_shows — initial rect, a window that moves mid-take, and the clamped out-of-buffer origin.
  • cursor_sample_tests::a_window_capture_reports_coordinates_inside_the_window — the bug itself.
  • cursor_sample_tests::a_full_screen_capture_reports_stream_coordinates — the no-regression case.
  • cursor_sample_tests::a_pointer_outside_the_window_is_reported_invisible — both directions (past the far edge, and negative).
  • cursor_sample_tests::nothing_is_emitted_before_the_format_is_known
  • cursor_sample_tests::the_content_rect_prefers_the_encoder_once_it_has_started

Also run:

  • cargo clippy --all-targets — no new warnings (the 6 existing ones are pre-existing dead-code notes).
  • npx tsc --noEmit and npx tsc -p tsconfig.test.json --noEmit — clean.
  • npm run lint — 14 warnings, identical to a clean main.
  • npm run test — 2045 pass / 40 fail, identical counts to a clean main (pre-existing jsdom localStorage setup failures in src/components/**, unrelated to this change; verified by stashing and re-running).
  • node scripts/check-docs.mjs — OK (31 files).
  • npm run build:native:linux — the release helper builds and its startup probe reports {"cursorMetadataSupported":true,"pipewireVersion":"1.0.5","event":"ready"}.

Manual verification on Ubuntu / GNOME / Wayland (mutter, AMD): the v1.10.0 release AppImage was extracted and run against the rebuilt helper via OPENSCREEN_LINUX_CURSOR_HELPER_EXE, recording a window. The cursor lands where the pointer actually was. Full-screen recording re-checked for regressions.

…nitor

mutter never renegotiates the format for a window stream: it pins the
stream to the window's monitor and carves the window out through
SPA_META_VideoCrop, which can move on any buffer as the window does. The
encoder already reads through that rect, so the file holds the window --
but the portal reports the pointer in STREAM pixels, measured from the
monitor's corner, and emit_sample normalised it against the negotiated
format. A monitor-relative position divided by monitor dimensions, then
painted onto window-sized footage: wrong origin and wrong scale, in every
window recording, from the first frame. A 640x480 window at (100, 50) on
1920x1080 put a pointer at the window's centre at 0.219, 0.269.

Capture now remembers the rect it actually read -- read_origin at the
committed size, so it follows a window that moves and keeps the clamp that
stops a shrunken window reading past the buffer -- and content_rect() is
the one place that answers "what does the file show". emit_sample takes
the origin off the position and reports that rect's dimensions, so the
accumulator's x/width lands in the space the compositor assumes.

`visible` moves with it. It was tested against the monitor, so a pointer
that had left the recorded window still reported visible: true.

Full-screen capture is byte-identical: no crop means the content rect is
the stream. A cursor-only session opens no encoder and keeps normalising
against the stream, which is right -- its video comes from Electron.

Not getopenscreen#511. That one is temporal (the video time-compresses under frame
drops while the cursor keeps wall-clock time, so the error grows across
the take); this one is spatial and constant. They stack.
@coderabbitai

coderabbitai Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Linux window capture draws the cursor at the wrong place - pointer is normalised against the monitor, not the window

1 participant