Symptom
x86 boot test hangs: thread 18 (tcp_socket_test) issues sys_read(fd=22) on a TCP
socket, blocks in TCP_BLOCK waiting for data, and is never rescheduled. The kernel
falls into a permanent idle loop (HLT forever, no further progress, test times out).
Confirmed pre-existing on main at a38ce4a4b2e4f1008f5421a0726067c97e954f46
(3/3 reproductions, identical signature/backtrace) — i.e. independent of and not
introduced by PR #470 PR-2 (fix/470-pr2-leaves@9319d4bb), which is otherwise fully
green (aarch64 prove-to-zero: clean-gate 300 + starved-gate 100, 0 faults, wired
92/92; x86 build clean, 0 warnings; Parallels 3/3). Branch fix/470-pr2-leaves
actually reproduces the hang less often than main (2/3) in local runs — consistent
with a scheduler/wakeup race rather than anything PR-2 touches.
Root cause (mechanism) — CONFIRMED
Scenario (a): the peer's TCP segment is produced but never delivered because
loopback delivery is pull-based, and nothing pumps it once the last runnable user
thread blocks in recv.
The localhost path enqueues packets into LOOPBACK_QUEUE
(kernel/src/net/mod.rs:234, log line "NET: Loopback detected, queueing packet for deferred delivery" at kernel/src/net/mod.rs:837) and only delivers them when
something calls net::drain_loopback_queue() (kernel/src/net/mod.rs:331).
That drain is currently called from specific socket syscalls (sendto/recvfrom/
connect/accept/write — see kernel/src/syscall/socket.rs:463,1475 and
kernel/src/syscall/handlers.rs:440,3341,3532,3710) and from the special
single-test idle loops (dns_test_only / blocking_recv_test /
nonblock_eagain_test at kernel/src/main.rs:819, :887, :958) — but it is
not called from the general kernel idle loop, idle_thread_fn
(kernel/src/main.rs:1751).
So: once the last user thread blocks in recv waiting on a socket, and the peer's
reply is sitting in LOOPBACK_QUEUE un-drained, the kernel enters the general idle
loop (not one of the special test-only idle loops) and nothing ever pumps the queue
again. The blocked thread is never woken. Permanent hang.
Fix scope — Small / low difficulty
Primary fix: call crate::net::drain_loopback_queue() from the general kernel
idle loop idle_thread_fn (kernel/src/main.rs, ~line 1751), exactly mirroring what
the special test-mode idle loops already do
(kernel/src/main.rs:819, :887, :958). This guarantees pending loopback packets
are delivered whenever the system goes idle, so a blocked recv-waiter gets woken.
Optional hardening: also drain inside the blocking HLT loops themselves — e.g.
TCP recv in kernel/src/syscall/handlers.rs (~lines 1272-1308), and similarly for
TCP accept / UDP recvfrom / TCP connect — so delivery isn't solely dependent on
reaching the top-level idle loop.
Files:
kernel/src/main.rs
kernel/src/syscall/handlers.rs
kernel/src/syscall/socket.rs
Root cause is known; this does not touch any Tier-1/Tier-2 prohibited files, and the
fix is a straightforward addition of an existing, already-used drain call to one more
call site (plus optional hardening at a couple more).
Status
This is a follow-up to be fixed after the current #470 (aarch64 leaf-frame
custody) work settles. Filed per operator ruling on PR #470 PR-2
(fix/470-pr2-leaves) review — the hang is orthogonal to PR-2's leaf-custody changes
and does not block that merge.
Symptom
x86 boot test hangs: thread 18 (
tcp_socket_test) issuessys_read(fd=22)on a TCPsocket, blocks in
TCP_BLOCKwaiting for data, and is never rescheduled. The kernelfalls into a permanent idle loop (HLT forever, no further progress, test times out).
Confirmed pre-existing on
mainata38ce4a4b2e4f1008f5421a0726067c97e954f46(3/3 reproductions, identical signature/backtrace) — i.e. independent of and not
introduced by PR #470 PR-2 (
fix/470-pr2-leaves@9319d4bb), which is otherwise fullygreen (aarch64 prove-to-zero: clean-gate 300 + starved-gate 100, 0 faults, wired
92/92; x86 build clean, 0 warnings; Parallels 3/3). Branch
fix/470-pr2-leavesactually reproduces the hang less often than main (2/3) in local runs — consistent
with a scheduler/wakeup race rather than anything PR-2 touches.
Root cause (mechanism) — CONFIRMED
Scenario (a): the peer's TCP segment is produced but never delivered because
loopback delivery is pull-based, and nothing pumps it once the last runnable user
thread blocks in
recv.The localhost path enqueues packets into
LOOPBACK_QUEUE(
kernel/src/net/mod.rs:234, log line"NET: Loopback detected, queueing packet for deferred delivery"atkernel/src/net/mod.rs:837) and only delivers them whensomething calls
net::drain_loopback_queue()(kernel/src/net/mod.rs:331).That drain is currently called from specific socket syscalls (
sendto/recvfrom/connect/accept/write— seekernel/src/syscall/socket.rs:463,1475andkernel/src/syscall/handlers.rs:440,3341,3532,3710) and from the specialsingle-test idle loops (
dns_test_only/blocking_recv_test/nonblock_eagain_testatkernel/src/main.rs:819,:887,:958) — but it isnot called from the general kernel idle loop,
idle_thread_fn(
kernel/src/main.rs:1751).So: once the last user thread blocks in
recvwaiting on a socket, and the peer'sreply is sitting in
LOOPBACK_QUEUEun-drained, the kernel enters the general idleloop (not one of the special test-only idle loops) and nothing ever pumps the queue
again. The blocked thread is never woken. Permanent hang.
Fix scope — Small / low difficulty
Primary fix: call
crate::net::drain_loopback_queue()from the general kernelidle loop
idle_thread_fn(kernel/src/main.rs, ~line 1751), exactly mirroring whatthe special test-mode idle loops already do
(
kernel/src/main.rs:819,:887,:958). This guarantees pending loopback packetsare delivered whenever the system goes idle, so a blocked recv-waiter gets woken.
Optional hardening: also drain inside the blocking HLT loops themselves — e.g.
TCP recv in
kernel/src/syscall/handlers.rs(~lines 1272-1308), and similarly forTCP accept / UDP recvfrom / TCP connect — so delivery isn't solely dependent on
reaching the top-level idle loop.
Files:
kernel/src/main.rskernel/src/syscall/handlers.rskernel/src/syscall/socket.rsRoot cause is known; this does not touch any Tier-1/Tier-2 prohibited files, and the
fix is a straightforward addition of an existing, already-used drain call to one more
call site (plus optional hardening at a couple more).
Status
This is a follow-up to be fixed after the current #470 (aarch64 leaf-frame
custody) work settles. Filed per operator ruling on PR #470 PR-2
(
fix/470-pr2-leaves) review — the hang is orthogonal to PR-2's leaf-custody changesand does not block that merge.