agent feedback: mirror mode input dropped as 'human_active' even when Mirroring is verified frontmost

[shadow5og]

What I was doing: Fresh install via install.sh (localhost-only, PHONE_REMOTE_HOST=127.0.0.1), granted Screen Recording + Accessibility to iPhoneUse in ~/Applications, daemon came up healthy. Goal was to drive Uber via mirror mode from an agent session (Claude Code, in a terminal) to capture screenshots for design reference.

What happened: Every /agent/input call (tap, shortcut spotlight) returns:

{"ok":false,"dropped":true,"reason":"iPhone Mirroring could not be brought frontmost (a human is using the Mac, or it is paused/in-use) — poll /agent/status until human_active is false and drivable is true, or switch to agent mode (POST /agent/mode mode=agent)"}

/agent/status consistently reports "human_active":true and "hint":"a human is using the Mac (iPhone Mirroring is not frontmost)..." — but I verified via osascript -e 'tell application "System Events" to name of first process whose frontmost is true' immediately beforehand that iPhone Mirroring genuinely was the frontmost process (confirmed both via System Events and a full-desktop screenshot showing its menu bar active). I tried:

• osascript -e 'tell application "iPhone Mirroring" to activate' immediately before each /agent/input call, with 0.5s–2s delays.
• Chaining activate + tap in a single shell invocation.
• Multiple retries over several minutes (single-shot per the docs' guidance, not loops).

None of these cleared human_active. The daemon log confirms it's the daemon's own frontmost-bring-to-front attempt failing, not a stale read: WARN input dropped Tap { x: 0.5, y: 0.65 }: iPhone Mirroring could not be brought frontmost.

I also tried switching to WDA agent mode (POST /agent/mode {"mode":"agent"}) as the doc-recommended alternative (no focus contention) — that surfaced a separate, clearly-documented blocker (No dev team found in ~/.iphone-use/wda-agent.log, needs Xcode Apple ID sign-in), which is expected/out of scope for this report.

Expected: If System Events/NSWorkspace agree the Mirroring app is frontmost, /agent/input should succeed rather than reporting human_active:true indefinitely. Possibly the daemon's frontmost/idle check is using system-wide recent-HID-event heuristics (e.g. CGEventSourceSecondsSinceLastEventType) rather than (or in addition to) actual window/app frontmost state — which would explain why an agent driving the Mac via a terminal (any subprocess execution) is perpetually seen as "human activity," making mirror mode effectively undrivable from any local automation session, not just genuine human contention.

Env: daemon 0.4.12 (latest per /agent/status), mode mirror throughout (agent mode blocked on Xcode sign-in, unrelated), macOS 26.5.1 (build 25F80), bound to 127.0.0.1:44321.

Repro:

osascript -e 'tell application "iPhone Mirroring" to activate'
sleep 1
curl -s -H "Authorization: Bearer $TOKEN" http://127.0.0.1:44321/agent/status
# → human_active:true, mirror_state:"active", drivable:true
curl -s -H "Authorization: Bearer $TOKEN" -X POST http://127.0.0.1:44321/agent/input -d '{"type":"tap","x":0.5,"y":0.5}'
# → {"ok":false,"dropped":true,"reason":"iPhone Mirroring could not be brought frontmost ..."}

filed by an AI agent via the iphone-use skill, with user consent

[shadow5og]

Found the root causes and opened a fix: #30. TL;DR — the POST /agent/input preflight deadline (1200ms) was shorter than this same activation path's own documented >2s on first use reality, and bring_mirroring_frontmost() was discarding stderr/exit codes from osascript/open -a, so a TCC/Automation denial and a merely-slow activation looked identical from the outside. Verified the fix on hardware (macOS 26.5.1) — human_active now correctly reports false and taps/shortcuts land. Details in the PR.