P0: E2E suite cannot gate at 10/15 passing flows

[RonenMars]

Baseline measured 2026-08-11. 11 pass, 4 fail, and the failing four are exactly the four this issue was filed about. No new breakage was hiding behind the familiar red.

Verified state

Full npm run test:e2e:mock on iPhone 17 Pro Max / iOS 26.1, against a Release build verified current by the #598 staleness guard. Maestro reported 4/15 Flows Failed.

Passing (11): launch, browse, bug6_bottom_bar_inset, codex_parity, settings_qr_scanner, server_drag_reorder, feat1_tree_drill_new_session, feat2_export_in_info_shelf, pty_turn_divider, voice_dictation, 07_conversation_scroll_gaps.

Failing (4):

Flow	Fails asserting	Timeout
session_lifecycle	hub-screen visible	8000
feedback_flow	settings-help-feedback-row visible	4000
05_chat_flow	first-session-card visible	10000
06_search_anchor	conversation-row-conv-search-anchor visible	8000

Every one dies on an assertConditionCommand for a visible element. None crashes, and none hits a driver fault. So these are testID or fixture problems rather than the XCUITest-infrastructure flake that has muddied past runs — the tractable kind.

Worth noting separately: session_lifecycle fails at hub-screen, which is setup. Its later steps have therefore never executed, so whatever else may be wrong with that flow is still unmeasured.

What this changes

This issue's premise was that red is the expected state, so new breakage is indistinguishable from the familiar four. That premise now has a measurement behind it: a named known-good set of 11 and a named failing 4, each with the exact assertion that fails.

Flows can now be fixed one at a time with the count as the signal.

Note on how this was measured

Two mistakes worth recording, because both produced confident wrong answers:

• The run was piped through tail, so the reported process exit code was tail's 0 rather than Maestro's non-zero. That briefly looked like a suite passing with four failures — a broken CI gate that does not exist.
• The same tail truncated the per-flow verdicts out of the log. The failing set was recovered from Maestro's screenshot-❌-…-(flow).png artifact filenames under e2e/_artifacts/debug/.

Do not pipe a suite run through tail when the per-flow result is the thing you need.

---

The mock Maestro suite passes 11 of 15 flows, and has done for long enough that red is the expected state.

That means new breakage is indistinguishable from the familiar four, so the suite cannot gate anything.

Verified state

test:e2e:mock in package.json runs 15 flows.

Four are known-failing for reasons documented in docs/e2e-remaining-work.md and docs/followups/mobile/06-mock-suite-remaining-failures.md: session_lifecycle, feedback_flow, 05_chat_flow and 06_search_anchor.

Depends on

This needs a build you can trust first — see the ensure-release-build.js staleness issue in this milestone. A pass set derived from a possibly-stale .app is a guess.

Done looks like

Either the four flows pass, or they are explicitly quarantined with a recorded reason, so that any red result is new information.

Fix the flows, not the app — and stop and report if a genuine product bug turns up rather than editing the flow around it.

Reference

docs/followups/repo-health/03-e2e-suite-signal.md and docs/followups/mobile/06-mock-suite-remaining-failures.md

feedback_flow — diagnosed, not yet fixed (2026-08-11)

The flow reaches Settings successfully; the failure screenshot shows the screen rendered, scrolled to the top. It then asserts settings-help-feedback-row visible and times out.

The element exists and is unconditional — app/settings.tsx:940, a plain TouchableOpacity with no gating, inside the screen's ScrollView (:440-954).

It is off-screen. The Appearance section between the top and it now includes a Language row, a Layout row, a Dark/Light toggle and a grid of nine theme cards. settings_qr_scanner passes on the same screen only because its target sits at :474, above the fold.

But scrolling does not reach it either. Tried scrollUntilVisible at timeout: 20000, speed: 80, visibilityPercentage: 60, centerElement: true. Maestro still reports "No visible element found".

That is the interesting part. The row is inside a real ScrollView and is not conditionally rendered, so a plain vertical scroll should find it. Candidate explanations, none yet confirmed:

• The swipe gesture is being captured by a horizontally-scrollable child — the Language selector and the theme grid both sit in the path.
• The ScrollView is not receiving the gesture at the coordinates Maestro swipes from.
• A nested scroll container is consuming it.

If it is the first or third, this is an app defect rather than a flow defect — a user dragging from the middle of the Appearance section would fail to scroll too. That crosses the line the brief draws ("fix the flows, not the app, unless a genuine product bug turns up — then stop and report").

Worth checking by hand before writing more Maestro: open Settings on a simulator, drag upward starting over the theme grid, and see whether the screen scrolls. If it does not, the bug is in the app and this flow is only the messenger.

The other three failures have not been diagnosed yet.

---

Progress — 2026-08-11 (2 of 4 fixed)

feedback_flow — fixed, PR #642 (1ab1cadd). Two defects: the target row was below the fold (and scrollUntilVisible cannot find it even fully tuned, while two plain swipes can), and behind that, a tap on feedback-diagnostics-toggle was reported COMPLETED while landing on the keyboard and typing a stray V into the description.

session_lifecycle — fixed, PR #645. The baseline's "fails at hub-screen" was misleading: setup passes, and the flow dies on the final hub-screen assert after - back. Browse is a presentation: 'modal' screen, iOS modals dismiss by dragging down, and - back is a no-op that Maestro still reports COMPLETED. Replaced with a downward swipe.

06_search_anchor — diagnosed, not fixed

Typing "wombat" reaches the server — the mock logs 3 /api/search requests — and e2e/fixtures/search-results.json does contain conv-search-anchor. The endpoint and the fixture are both correct.

The testID is correct too: conversation-row-<id> is rendered by the Classic list (components/conversation/ConversationList.tsx:68) as well as Tree and Hub, so the repo's default sessionsLayout: 'classic' is not the problem.

What the failure screenshot actually shows is the sessions empty state — "No sessions" — with the query present in the input. So the results are being returned but rendered somewhere other than the pane on screen; the visible pane is sessions, and the match is a conversation.

Next step for whoever picks this up: determine which surface search results render into (a History/conversations pane rather than sessions), and whether the flow needs to switch to it before asserting. Do not lengthen the timeout — the data arrives, it is displayed elsewhere.

A measurement trap worth avoiding

A first attempt at this flow appeared to show the mock server crashing with Assertion failed: (advance > 0), function substitute, file process.c. That was not a repo bug: a leaked mock-server.js from an earlier run still held port 7071, the new instance died with EADDRINUSE, and the native assertion was a secondary artifact.

Run pkill -f mock-server.js and check lsof -ti:7071 before trusting a run.

05_chat_flow

Untouched. Still failing at first-session-card.

06_search_anchor — blocked on a product bug, not fixable as a flow

Traced to the end 2026-08-11. The flow cannot pass in the default configuration, because it asserts behaviour the app does not currently have.

app/index.tsx:510 is the only place server-search results reach a list, and it sits in the non-merged branch of the ternary at :462. stores/settings.ts:133 defaults mergeChats: true, so MergedClassicList renders instead and never receives convSearchData. The merged path filters locally over loaded items (:641-654), and conv-search-anchor is deliberately absent from the loaded set.

The request is made and discarded — the mock logged 3 GET /api/search hits during the failing run.

Filed as #646.

Three fixes were tried and rejected, recorded so nobody repeats them:

• Longer timeout — the data arrives, so waiting cannot help.
• Tap hub-history-tab — that tab only exists when mergeChats is false; the tap fails outright.
• Add conv-search-anchor to conversations.json — the app never calls GET /api/conversations, only /api/conversations/count, so the fixture is never read. Reverted.

Recommendation: leave this flow failing until #646 is decided. It is currently the tracker's most accurate signal — a correct assertion against a real gap. Making it green by forcing mergeChats: false would test a non-default configuration and hide the defect.

That puts the honest target for this issue at 13 passing, 2 failing, with both remaining failures explained: 06_search_anchor blocked on #646, 05_chat_flow still to be attempted.

---

Re-measured 2026-08-12 — 10 pass, 5 fail, and the earlier baseline was an overcount

Full npm run test:e2e:mock against 86eeb2a1 with a freshly built Release app, iPhone 17 Pro Max / iOS 26.1.
Verdicts read from Maestro's own [Passed]/[Failed] lines this time, not inferred.

Passing (10): launch, browse, server_drag_reorder, bug6_bottom_bar_inset, pty_turn_divider, feat1_tree_drill_new_session, settings_qr_scanner, feedback_flow, 05_chat_flow, 07_conversation_scroll_gaps.

Failing (5):

Flow	Fails asserting	Tracked as
session_lifecycle	hub-screen visible	#661
feat2_export_in_info_shelf	"Resume Session" visible	fixed in PR #658
codex_parity	"Resume Session" visible	fixed in PR #658
voice_dictation	chat-mic-button visible	#662
06_search_anchor	search-match-nav visible	#663

05_chat_flow now passes, fixed in #648.
feedback_flow stays fixed from #642.

The 2026-08-11 baseline above listed three of these as passing, and that was wrong

codex_parity, feat2_export_in_info_shelf and voice_dictation are recorded as passing in the 2026-08-11 section, and all three fail now.

That is not a regression.
The <Play weight="fill"/> icon whose accessibility text breaks the two Resume Session assertions was added on 2026-08-02 in #510, and is present at 1ab1cadd, the commit the 08-11 baseline was measured against.

The likelier explanation is the measurement itself.
That run was piped through tail, which truncated the per-flow verdicts, and the failing set was reconstructed from screenshot-❌-… artifact filenames.
That method observes only failures — the passing list was everything left over, so any flow failing without leaving a ❌ screenshot was silently counted as a pass.

Treat the 11/15 figure as unverified. 10/15 is measured directly.

Breakdown now tracked as individual issues

• P1: ensure-release-build.js can never be satisfied by the fix it prescribes #660 (P1) — ensure-release-build.js can never be satisfied by the fix it prescribes, which blocks running this suite the documented way at all. Worth clearing first.
• P2: session_lifecycle fails the mock suite at the final hub-screen assertion #661 — session_lifecycle.
• P2: voice_dictation fails because the composer shows send, not mic #662 — voice_dictation.
• P2: 06_search_anchor fails at search-match-nav after the hub search fix #663 — 06_search_anchor, now failing one step later than before.
• P2: hub session cards can be tapped before layout, failing flows intermittently #664 — hub cards tappable before layout, an intermittent race that can fail any flow entering a session.
• P3: no E2E coverage that a running session shows the thinking indicator #665 — no coverage that a running session shows the thinking indicator, after the unreachable assertion was removed in test(e2e): mock the session input endpoint and fix the chat-flow assertions #648.

On #646, referenced in the section above

The hub half is fixed in PR #657: merged-mode search now sources conversations from the server, and 06_search_anchor gets past conversation-row-conv-search-anchor and opens the conversation.

#646 stays open for the remaining half — the client still re-filters server results on title and preview, so a conversation matched on message body is fetched and then discarded.
That half is specified in docs/prompts/fix-merged-search-client-refilter.md, and the shared e2e fixture cannot detect it.