x86: boot thread wedges in the RING3_SMOKE disk read while holding the process-manager lock, blocking every first ring-3 entry

[ryanbreen]

Summary

On x86_64 with --features boot_tests,testing,external_test_bins, the boot thread stops making
progress inside test_exec::test_direct_execution()'s get_test_binary("hello_time") disk read
(kernel/src/test_exec.rs:37) in every boot — CONCURRENCY TEST: Loading hello_time.elf
(the next statement) is printed 0 times in 22/22 observed boots, passing and failing alike.

In roughly a third of boots, a process-manager guard is live when that happens and is never
released. Every subsequent first ring-3 entry then cannot obtain the manager, so no user thread can
ever be dispatched and the boot never reaches USERSPACE TEST COMPLETE / TEST RUNNER: All tests passed.

This is pre-existing and independent of the #470 PR-3 free path: no frame is freed, no PTE is
cleared and no custody record is involved — it is a lock held across a block-device wait.

Evidence (beast, Incus VM breenix-x86, TCG, branch fix/470-pr3-x86-free-path)

22 full boots. Every one emits the custody gates and [PT_RETIRE_COHORT:x86:...]; none shows a
0x14 or 0x15 fault. 17 complete, 5 wedge. In each wedge the serial log ends in an endless cycle of

kernel::tls: Set FS base to 0x9c000
kernel::interrupts::context_switch: First run: thread 140 entering userspace
kernel::interrupts::context_switch: setup_first_userspace_entry: thread 140

for threads 140..149, ~410 times over. A temporary diagnostic build (not committed) that logs the
first 12 aborts with the manager-owner snapshot gives, for every one of them:

DIAG abort#0 thread 140: process manager unavailable resume=0 pm_owner=Some((0, 0))
DIAG abort#1 thread 141: process manager unavailable resume=0 pm_owner=Some((0, 0))
...

pm_owner = (cpu 0, tid 0) — a live guard attributed to the boot/idle thread, held for the rest of
the boot. process::try_manager() is a plain try_lock, so this is a genuine held lock, not a
lookup failure.

The stall point is the same in passing boots — the boot thread is abandoned there in all of them —
so what distinguishes a wedged boot is only whether the guard is live when the first dispatch is
attempted.

Related

• The x86 Completion::wait_timeout_inner() "boot-thread spin path" has no enforced deadline
  (kernel/src/task/completion.rs; the computed deadline is only honoured on aarch64), so a lost
  block-device completion spins forever instead of returning an error. That is the most likely
  reason the disk read never returns.
• Lock-inversion class already tracked in aarch64 exec path inverts PM/SCHEDULER lock order (latent — try_lock-only today, deadlocks if a blocking site is added) #527.
• Surfaced while fixing x86: not-present fault (0x14) at user RIP 0x4000e2bc on first entry to userspace in http_test #553 (first ring-3 entry could IRETQ to ring 3 without its CR3). That fix
  makes this condition fail closed — the thread is no longer sent to ring 3 on the kernel PML4 —
  but it cannot manufacture the missing manager, so the boot still wedges. Before the fix the same
  condition produced a 0x14 fault, a killed process and a wedge.

What a fix needs to decide

1. Whether the process-manager lock may be held across a block-device wait at all (the contained
   fix is to narrow the scope so it cannot be), and
2. whether the x86 completion spin path should enforce its deadline like aarch64 does, turning an
   infinite hang into a surfaced error.

Both are outside the scope of the PR-3 custody work and neither touches the allocator or the free
path.