Skip to content

admin: stream /state as the walk produces it - #633

Merged
afrind merged 1 commit into
relay/walk-state-on-owning-executorfrom
admin/stream-state-as-walk-produces-it
Aug 26, 2026
Merged

admin: stream /state as the walk produces it#633
afrind merged 1 commit into
relay/walk-state-on-owning-executorfrom
admin/stream-state-as-walk-produces-it

Conversation

@afrind

@afrind afrind commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

/state serialized the whole document before sending a byte, so a relay with a large namespace tree or cache held the entire response in memory, and the client waited for the last service to be walked before seeing the first.

The walk now writes into an AsyncPipe that the admin EVB drains into a chunked response. Producer and consumer run concurrently: the producer migrates across relay executors, while the consumer stays on the admin EVB where sendBody is thread-safe and parks on the EgressGate whenever the transaction cannot write. A slow client therefore leaves the unsent bytes in the pipe rather than relocating them into proxygen's egress queue, where nothing would account for them. The walk itself never stalls: it is synchronous and never yields its executor, so the consumer cannot push back on it. StreamBudget bounds what that can cost instead, capping what may pile up in the pipe and failing the response past it. Headers wait for the first chunk, so a walk that throws before producing output is still reported as a 500 rather than an aborted 200.

Both visitors gain alive(), checked at section boundaries, so a client that hangs up stops the walk rather than formatting the rest for nobody.

StateStreamTest covers the visitors and chunk boundaries, StateResponseTest the consumer, budget and gate, and test_admin_state.sh the framing end to end.


This change is Reviewable

@akash-a-n akash-a-n left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akash-a-n reviewed 16 files and all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on afrind).

/state serialized the whole document before sending a byte, so a relay with a
large namespace tree or cache held the entire response in memory, and the
client waited for the last service to be walked before seeing the first.

The walk now writes into an AsyncPipe that the admin EVB drains into a chunked
response. Producer and consumer run concurrently: the producer migrates across
relay executors, while the consumer stays on the admin EVB where sendBody is
thread-safe and parks on the EgressGate whenever the transaction cannot write.
A slow client therefore leaves the unsent bytes in the pipe rather than
relocating them into proxygen's egress queue, where nothing would account for
them. The walk itself never stalls: it is synchronous and never yields its
executor, so the consumer cannot push back on it. StreamBudget bounds what that
can cost instead, capping what may pile up in the pipe and failing the response
past it. Headers wait for the first chunk, so a walk that throws before
producing output is still reported as a 500 rather than an aborted 200.

Both visitors gain alive(), checked at section boundaries, so a client that
hangs up stops the walk rather than formatting the rest for nobody.

StateStreamTest covers the visitors and chunk boundaries, StateResponseTest the
consumer, budget and gate, and test_admin_state.sh the framing end to end.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@afrind
afrind force-pushed the admin/stream-state-as-walk-produces-it branch from 4a3ac9e to bf0c68e Compare August 26, 2026 16:18
@afrind
afrind merged commit bffca92 into main Aug 26, 2026
9 of 15 checks passed
@afrind
afrind deleted the admin/stream-state-as-walk-produces-it branch August 26, 2026 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants