fix: keep HTTP session open while visit() and streaming query generators are consumed

[thomasht86]

Summary

Vespa.visit() and Vespa.query(streaming=True) returned lazy generators from inside a with VespaSync(...) block, so the underlying httpr.Client was closed before the first request was ever sent. This went unnoticed because httpr<=0.6.0 treated close() as a no-op and a "closed" client kept working. httpr 0.7.0/0.7.1 released the pool on close() and raised httpr.ClientClosed on any later request, which broke every app.visit(...) call:

httpr.ClientClosed: Cannot send a request, as the client has been closed.

Status on the httpr side: 0.7.1 is yanked, 0.7.0 was never published, and 0.7.2 reopens a closed client on use while emitting httpr.ClientReopenedWarning (a ResourceWarning subclass), so released pyvespa versions keep working. This PR fixes the latent pyvespa bug regardless of httpr version, and no version bound is changed.

Changes

• Vespa.visit() now delegates to _visit_with_session(), which reference-counts the session: the outer generator and every slice generator hold one reference, and the client is closed when the last one is exhausted or garbage collected. This also keeps patterns like list(app.visit(...)) or handing each slice to a worker thread working, which a plain yield from inside the with block would not.
• Vespa.query(streaming=True) now runs in _query_streaming_with_session(), which keeps the session open until the stream is drained.
• Regression tests in tests/unit/test_application.py (TestSessionOutlivesLazyGenerators) use a mock client that raises after close(). They fail on master and pass with this change.
• uv.lock bumps httpr to 0.7.2 so the uv sync based cloud jobs exercise the current release (the pip based jobs already resolve to it).

Testing

See PR comments for the full runs. In short, with httpr==0.7.1 master fails 7 Docker and 2 Cloud integration tests via visit, all passing with this fix; with httpr==0.7.2 the unit suite (696 tests) and the same 7 Docker tests pass, with no ClientReopenedWarning emitted.

🤖 Generated with Claude Code

[thomasht86]

Context from the httpr side (posted on behalf of @thomasht86):

httpr 0.7.2 and pyvespa: what happened and what's next

httpr 0.7.0 made Client.close() actually release the connection pool and raise ClientClosed on any later request. That exposed a latent pattern in every released pyvespa: Vespa.visit() and Vespa.query(streaming=True) return lazy generators from inside a with VespaSync(...) block, so the client is closed before the first request runs. It worked for a year only because close() was a no-op.

What was done on the httpr side:

• httpr 0.7.2 reopens a closed client on use, the way requests.Session does, and emits httpr.ClientReopenedWarning, a ResourceWarning subclass. It is silent for end users and visible in test runs. close() still releases the pool immediately. warnings.simplefilter("error", httpr.ClientReopenedWarning) restores the strict behaviour if you ever want it.
• 0.7.1 is yanked and 0.7.0 was never published, so unpinned installs skip the breaking versions. Published pyvespa releases allow any httpr>=0.4.0 and are covered by 0.7.2.
• Verified against unmodified pyvespa master: the unit suite passes with the reopen warning promoted to an error, and the Docker integration tests that drive visit() pass with exactly one warning each, confirming the reopen path covers the real regression.
• httpr CI now has a downstream-pyvespa job that runs pyvespa master's unit suite against every httpr PR, so a change like this cannot reach a release unnoticed again.

Next steps on the pyvespa side:

1. Merge this PR and cut a patch release. The fix is correct regardless of httpr version, and the regression tests are the durable part. No httpr version bound is needed, so httpr>=0.4.0 stays.
2. Treat the warning as an error in pyvespa CI with -W error::httpr.ClientReopenedWarning (considered and skipped: the unit regression tests and the downstream pyvespa job in httpr CI cover this without coupling pyvespa CI to an httpr class name). On the fixed branch these stay silent, and the filter turns any future generator-outlives-client regression into a hard failure.
3. Run the Cloud integration suite on the fix to close the loop (in progress, results will be posted here).

[thomasht86]

Verification with httpr==0.7.2 and -W error::httpr.ClientReopenedWarning, on this branch:

• tests/unit/: 696 passed.
• The 7 Docker integration tests that failed on master with 0.7.1 (TestMsmarcoApplication::test_execute_data_operations, the three TestQaApplication tests, both TestRetryApplication tests, TestDocumentExpiry::test_document_expiry): 7 passed, no reopen warning raised.
• Cloud suite: running, will post when done.

[thomasht86]

Cloud integration suite on this branch with httpr==0.7.2 (from the lockfile), run locally with no other deployment in flight:

tests/integration/test_integration_vespa_cloud.py: 20 passed, 4 skipped, 0 failed (43 min). On master with httpr 0.7.1 the same suite failed TestMsmarcoApplication::test_execute_data_operations and TestRetryApplication::test_retry with httpr.ClientClosed.

Note: the earlier Deployment was aborted, probably by a newer deployment failures, both in the GitHub integration-cloud run on this PR and in a concurrent local run, were the two runs deploying to the shared pyvespa-integration application at the same time. That workflow ran on the PR only because an interim commit touched its workflow file; that change has since been dropped.