Skip to content

Bound concurrent transaction_mode => 'new' launches - #322

Open
pinodeca with Copilot wants to merge 4 commits into
mainfrom
copilot/add-admission-control-new-launches
Open

Bound concurrent transaction_mode => 'new' launches#322
pinodeca with Copilot wants to merge 4 commits into
mainfrom
copilot/add-admission-control-new-launches

Conversation

Copilot AI commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

df.start(..., transaction_mode => 'new') opened an extra loopback backend per caller with no cross-session cap, so bursty submissions could drive backend growth toward max_connections. This change adds database-wide admission control, bounded rejection, and explicit coverage/documentation for the supported concurrency envelope.

  • Admission control for 'new' launches

    • Adds a cluster-wide cap on concurrent loopback launch sessions before the extra connection is opened.
    • Uses session-level advisory-lock slots so the limit applies across PostgreSQL backends, not just within one process.
    • Introduces:
      • pg_durable.max_new_transaction_starts (default 2)
      • pg_durable.new_transaction_start_timeout (default 5s)
  • Sharper failure modes and diagnostics

    • Rejected launches now fail with an actionable admission-timeout error that includes the configured limit and wait duration.
    • Loopback-session failures are distinguished from admission failures:
      • connection/open failure
      • lock_timeout
      • statement_timeout
    • Tags loopback launch sessions with a dedicated application_name to make them identifiable in backend inspection.
  • No partial state on rejected launches

    • Admission happens before opening the loopback session, so timed-out/rejected 'new' launches do not leave behind partial df.instances, df.nodes, or runnable engine state.
    • transaction_mode => 'caller' behavior remains unchanged.
  • Concurrency coverage

    • Adds a multi-session E2E test that launches concurrent 'new' starts via separate PostgreSQL client sessions and proves:
      • the configured launch cap is not exceeded
      • excess callers time out with the expected error
      • rejected launches leave no persisted instance rows or orphaned work
  • Docs / operating envelope

    • Documents the new launch budget separately from worker SQL-execution capacity.
    • Updates the connection-budget guidance and reinforces that per-row / high-fan-out call sites remain unsafe unless validated against the configured cap.
-- New admission-control knobs for transaction_mode => 'new'
pg_durable.max_new_transaction_starts = 2
pg_durable.new_transaction_start_timeout = 5

SELECT df.start(
  'SELECT 1',
  'audit-launch',
  NULL,
  'new'
);

Copilot AI linked an issue Jul 30, 2026 that may be closed by this pull request
6 tasks
Copilot AI and others added 2 commits July 30, 2026 20:43
Co-authored-by: pinodeca <32303022+pinodeca@users.noreply.github.com>
Co-authored-by: pinodeca <32303022+pinodeca@users.noreply.github.com>
Copilot AI changed the title [WIP] Add admission control for concurrent 'new' launches Bound concurrent transaction_mode => 'new' launches Jul 30, 2026
Copilot AI requested a review from pinodeca July 30, 2026 21:09
@pinodeca
pinodeca marked this pull request as ready for review July 31, 2026 00:37
Add an E2E assertion that a transaction_mode => 'new' launch which acquires
an admission slot and then fails inside the loopback session still releases
its slot, plus clarifying comments on the advisory-lock namespace and the
poll-loop interrupt behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bound concurrent transaction_mode => 'new' launches

2 participants