fix: release connection-level flow control accounting for discarded buffered data - #1259
Open
pjfanning wants to merge 1 commit into
Open
fix: release connection-level flow control accounting for discarded buffered data#1259pjfanning wants to merge 1 commit into
pjfanning wants to merge 1 commit into
Conversation
…uffered data Motivation: `totalBufferedData` feeds the connection-level flow controller, which emits a WINDOW_UPDATE only while `outstanding + buffered` stays below half of `incoming-connection-level-buffer-size`. It is incremented for every DATA frame received but was not decremented on the three paths that discard a non-empty buffer: - `IncomingStreamBuffer.onRstStreamFrame` cleared the buffer after the peer reset the stream, - `IncomingStreamBuffer.onDownstreamFinish` cleared it when the application cancelled the entity stream, - `CollectingIncomingData.onRstStreamFrame` was a no-op even though the bytes collected so far had been counted. A peer that sends data the handler does not read and then resets the stream therefore ratchets `totalBufferedData` up permanently. Once the leaked total reaches half the configured buffer size the server stops replenishing the connection window, it drains to zero and every stream on that connection stalls. Modification: Add `IncomingStreamBuffer.discardBuffer()`, which subtracts what is still buffered from `totalBufferedData` before clearing it, and use it on both discard paths. Subtract the collected bytes in `CollectingIncomingData.onRstStreamFrame` for the same reason. Result: Resetting or cancelling a stream releases the connection-level window its buffered data reserved, so the connection keeps being replenished and no longer stalls. Tests: - sbt "http2-tests/testOnly org.apache.pekko.http.impl.engine.http2.Http2ServerSpec org.apache.pekko.http.impl.engine.http2.Http2ClientSpec org.apache.pekko.http.impl.engine.http2.Http2ClientServerSpec" - pass (174 tests); a new test buffers request data the handler never reads and resets the stream six times over, with the connection-level buffer size lowered to the initial window so the effect is reached quickly, then asserts the connection still accepts a request. Verified it fails with the fix stashed (the peer runs out of connection window). - sbt http-core/mimaReportBinaryIssues - pass (internal impl.engine.http2 change, no public API). References: None - releases buffered-data accounting when a stream is reset or cancelled
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
totalBufferedDatafeeds the connection-level flow controller, which emits a WINDOW_UPDATE only whileoutstanding + bufferedstays below half ofincoming-connection-level-buffer-size:It is incremented for every DATA frame received (
ReceivingData.handle), but was not decremented on the three paths that discard a non-empty buffer:IncomingStreamBuffer.onRstStreamFrame— clearedbufferafter the peer reset the stream;IncomingStreamBuffer.onDownstreamFinish— cleared it when the application cancelled the entity stream;CollectingIncomingData.onRstStreamFrame— a no-op, even though the bytes collected so far had already been counted.A peer that sends data the handler never reads and then resets the stream therefore ratchets
totalBufferedDataup permanently. Once the leaked total reaches half the configured buffer size, the server stops replenishing the connection window, it drains to zero, and every stream on that connection stalls. Note the RST_STREAM throttle does not prevent this — even a throttle-compliant reset rate leaks the counter.Modification
Add
IncomingStreamBuffer.discardBuffer(), which subtracts what is still buffered fromtotalBufferedDatabefore clearing it, and use it on both discard paths. Subtract the collected bytes inCollectingIncomingData.onRstStreamFramefor the same reason.Result
Resetting or cancelling a stream releases the connection-level window its buffered data reserved, so the connection keeps being replenished and no longer stalls.
Tests
sbt "http2-tests/testOnly org.apache.pekko.http.impl.engine.http2.Http2ServerSpec org.apache.pekko.http.impl.engine.http2.Http2ClientSpec org.apache.pekko.http.impl.engine.http2.Http2ClientServerSpec"— pass (174 tests). A new test buffers request data the handler never reads and resets the stream six times over, withincoming-connection-level-buffer-sizelowered to the initial window so the effect is reached quickly, then asserts the connection still accepts a request. Verified it fails with the fix stashed (the peer runs out of connection window).sbt http-core/mimaReportBinaryIssues— pass (internalimpl.engine.http2change, no public API).scalafmton the changed files — clean.References
None - releases buffered-data accounting when a stream is reset or cancelled
🤖 Generated with Claude Code