[core] Pin correlation-id draw order to event-log order by VaguelySerious · Pull Request #3700 · vercel/workflow

VaguelySerious · 2026-08-20T20:41:43Z

(AI) Runtime refactor pinning correlation-id draw order to event-log order, the scheduler-side alternative to call-site addressing (#3179) for the residual CORRUPTED_EVENT_LOG class that survives #3554. Default on in this PR (WORKFLOW_LOG_ORDER_DRAWS=0 opts out). Based on main, which now carries #3696's blocked-branch scenario: that scenario reproduces the corruption on unfixed code at 6/6 (vercel), 6/6 (local), 3/6 (postgres) attempts, #3179's arm measured clean with call-site ids, and this PR's event-log-race-repro lane measures the draw-order fix with ids left positional.

Problem

Correlation ids are ordinals of one per-run draw sequence, so the id an entity gets depends on every draw before it. #3554 pinned delivery retirement to log order, but a delivery's continuation still needs a path-dependent number of microtask hops to reach its next draw, and the discipline let later deliveries resolve mid-cascade: a deferring delivery yielded exactly one macrotask, a delivery consumed after an earlier barrier deregistered saw an empty gate set and never yielded at all, and two deliveries waiting concurrently broke the tie by timer arrival. Draw order was therefore a function of arrival, and arrival depends on how much log a replay loaded. Two concurrent replays holding different-length dense prefixes could bind one ordinal to two entities (step_ vs wait_ in the five 2026-08-20 production failures), and every later replay dies on an unconsumable event.

What this changes

awaitEarlierDeliveries gains a flag-gated quiescence fixpoint that runs on every branch-deciding delivery, not only deferring ones. A delivery resolves to the workflow only when, over a full (promise-queue drain + macrotask) turn:

no new correlation-id draws were made and no hydration is in flight (every earlier continuation has run as far as it can), and
no lower-index armed barrier is still registered (no committed earlier delivery is still pending — the tie-break that timer order used to decide).

Unarmed buffered hook payloads keep their claim-driven handover (the gatesOn skip is untouched): a claim happens at a body position, which is already a function of the prefix, and gating steps on unclaimed payloads is what the pre-#3554 harness proved loses the ordering entirely once the idle safety net has to fire.

Draw order then becomes a pure function of the dense log: extending the log can only append draws, never renumber them, so concurrent writers' duplicate creates carry identical ids and dedup (#3381) absorbs them.

The context exposes a monotone mintCount as the quiescence progress metric; lightweight test contexts without it degrade to single-turn quiescence.

Tests

log-order-draws.test.ts: the blocked-branch production shape through runWorkflow, nothing hardcoded. Flag off (control): a dense prefix ending before a sibling's launch completion rebinds an ordinal from a finalize step to the sibling's watchdog wait. Flag on: every shared correlation id keeps its binding under pairwise extension AND across a sweep of every dense prefix of the log; replay is deterministic per prefix.
Full unit suite green in both modes (2213 passed each way), including the delivery-sensitive suites (storm-log-replay, delivery-barrier-coverage, step-delivery-ordering, hook-sleep-interaction, retained-vm-loop, hop-count). No fixture had to be pinned or rewritten: where arrival order was stable, log order agrees with it; the modes differ only where arrival order was load-dependent, which is the bug.
delivery-barrier-coverage.test.ts gains a termination pin for the turnstile's riskiest shape: a step delivery spinning above an armed wait that is itself parked behind an unclaimed buffered payload. Termination there rests on resolvesOnItsOwn excluding the spinner from hasParkedCommittedDelivery, which keeps the idle safety net's gate reachable so it can retire the payload and open the chain. The test replays that log in both flag states; a turnstile or idle-predicate change that breaks the argument deadlocks it instead of only hanging e2e lanes.

Costs and open questions

Replay latency: one event-loop turn per branch-deciding delivery, more when continuations genuinely overlap. Review found the original setTimeout(0) turn cost ~1.1ms per delivery on sequential replays (Node clamps it); the turn is now setImmediate (~15-20µs), and a 100-step sequential replay measures 69.4ms flag-on vs 67.8ms flag-off (median of 5). The production storm-log replay is 30.9-31.3s in both modes, the hook e2e subset against a local prod server is timing-identical, and the unit suite total is unchanged. The benchmark lane's numbers on the current head are the remaining authoritative check.
Engine scope: this applies to the default Node VM engine only. WORKFLOW_RUNTIME=quickjs has its own event feed and correlation-id sequence (quickjs-runtime.ts mints via __generateUlid and never touches the barrier registry), so the setting is inert there — and the repro lane measures the node engine.
Live-side residual: this pins replay-driven scheduling. A resolution acted on in-process before its commit lands (and before a lower-slot event committed by another actor) is not reordered by this discipline; the orchestrator's iteration reload makes that window small, and the repro lane measures whether it matters in practice.
Scheme migration: draw order changes for logs written under arrival order, so enabling/disabling with runs in flight is only safe under deployment pinning (true on Vercel via skew protection). Documented in runtime-tuning.mdx.

Relation to #3179

Complementary, not competing: this fixes the schedule (covers waits and every same-kind entity, closing the per-kind residual call-site ids leave), #3179 fixes the identity (robust even if the scheduler drifts again). Either alone should clear the repro; together they are defense in depth. The repro lanes on the three PRs are the comparison.

🤖 Generated with Claude Code

…OW_LOG_ORDER_DRAWS A branch-deciding delivery now resolves to the workflow only after every earlier-in-log delivery's continuation has quiesced (no new draws, no hydration in flight, over a full macrotask turn) and no lower-slot armed barrier is still undelivered. Draw order becomes a pure function of the dense event log, stable under prefix extension, so concurrent replays holding different-length prefixes mint compatible correlation ids and their duplicate creates dedup instead of colliding (CORRUPTED_EVENT_LOG). The single post-deferral macrotask yield this replaces was arrival-order: a later delivery consumed after an earlier barrier deregistered saw an empty gate set and resolved mid-cascade, and two deliveries quiescing concurrently broke the tie by timer order. Buffered hook payloads keep their claim-driven handover (the gatesOn skip is unchanged): a claim is body-position determined, already a function of the prefix. Offline regression: the blocked-branch production shape replayed across every dense prefix, flag off (control, rebinds an ordinal step->wait) and flag on (every shared id keeps its binding). Full unit suite green in both modes. Default on for the repro A/B, mirroring the call-site-ids arm. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

changeset-bot · 2026-08-20T20:41:50Z

🦋 Changeset detected

Latest commit: 8e045a6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages

Name	Type
workflow	Minor
@workflow/core	Minor
@workflow/world-testing	Patch
@workflow/builders	Patch
@workflow/cli	Patch
@workflow/next	Patch
@workflow/nitro	Patch
@workflow/vitest	Patch
@workflow/web-shared	Patch
@workflow/web	Patch
@workflow/astro	Patch
@workflow/nest	Patch
@workflow/rollup	Patch
@workflow/sveltekit	Patch
@workflow/vite	Patch
@workflow/nuxt	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

vercel · 2026-08-20T20:41:53Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
example-nextjs-workflow-turbopack	Ready	Preview, v0	Aug 21, 2026 6:34pm
example-nextjs-workflow-webpack	Ready	Preview, v0	Aug 21, 2026 6:34pm
example-workflow	Ready	Preview, v0	Aug 21, 2026 6:34pm
workbench-astro-workflow	Ready	Preview, v0	Aug 21, 2026 6:34pm
workbench-express-workflow	Ready	Preview, v0	Aug 21, 2026 6:34pm
workbench-fastify-workflow	Ready	Preview, v0	Aug 21, 2026 6:34pm
workbench-hono-workflow	Ready	Preview, v0	Aug 21, 2026 6:34pm
workbench-nestjs-workflow	Ready	Preview, v0	Aug 21, 2026 6:34pm
workbench-nitro-workflow	Ready	Preview, v0	Aug 21, 2026 6:34pm
workbench-nuxt-workflow	Ready	Preview, v0	Aug 21, 2026 6:34pm
workbench-python-workflow	Ready	Preview, v0	Aug 21, 2026 6:34pm
workbench-sveltekit-workflow	Ready	Preview, v0	Aug 21, 2026 6:34pm
workbench-tanstack-start-workflow	Ready	Preview, v0	Aug 21, 2026 6:34pm
workbench-vite-workflow	Ready	Preview, v0	Aug 21, 2026 6:34pm
workflow-docs	Ready	Preview, v0	Aug 21, 2026 6:34pm
workflow-swc-playground	Ready	Preview, v0	Aug 21, 2026 6:34pm
workflow-tarballs	Ready	Preview, v0	Aug 21, 2026 6:34pm
workflow-web	Ready	Preview, v0	Aug 21, 2026 6:34pm

github-actions · 2026-08-20T20:42:02Z

🧪 E2E Test Results

❌ Some tests failed

❌ Failed E2E Tests

▲ Vercel Production (241 failed)

astro-node (11 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

astro-quickjs (9 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands

example-node (9 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

example-quickjs (11 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

express-node (9 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands

express-quickjs (12 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

fastify-node (10 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands

fastify-quickjs (10 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

hono-node (7 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting

hono-quickjs (9 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

nest-node (9 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting

nest-quickjs (11 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-turbopack-node (5 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes fire-and-forget: void setAttributes lands without awaiting

nextjs-turbopack-quickjs (9 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-webpack-node (9 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-webpack-quickjs (11 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nitro-node (7 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes fire-and-forget: void setAttributes lands without awaiting

nitro-quickjs (11 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

nuxt-node (8 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

nuxt-quickjs (11 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

sveltekit-node (5 failed):

hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes fire-and-forget: void setAttributes lands without awaiting

sveltekit-quickjs (11 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

tanstack-start-node (8 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

tanstack-start-quickjs (10 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

vite-node (8 failed):

concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

vite-quickjs (11 failed):

hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0JSVY020GW8FTGZZ6B94BYG | 🔍 observability
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0JSV2C20GSPE1D474AQ5VTP | 🔍 observability
hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0JSWV530GS39VEYYXJ460TA | 🔍 observability
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0JSXSEY0GYS1X4SBW0VFHK6 | 🔍 observability
setAttributes start: initial attributes are seeded on run creation
setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
setAttributes fire-and-forget: void setAttributes lands without awaiting
setAttributes Promise.all of disjoint-key writes: every key lands
setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

24 flaky tests

abortHookOrderingWorkflow [hook-first-abort-first]: hook.then → addEventListener → abort() → resumeHook (sveltekit)
abortParallelWorkflow: abort cancels all parallel steps (nextjs-webpack)
concurrent hook token conflict - two workflows cannot use the same hook token simultaneously (sveltekit)
deploymentId: 'latest' is a no-op in non-Vercel worlds (nitro)
deploymentId: 'latest' is a no-op in non-Vercel worlds (tanstack-start)
hookCleanupTestWorkflow - hook token reuse after workflow completion (sveltekit)
hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (sveltekit)
Promise.all of disjoint-key writes: every key lands (example)
Promise.all of disjoint-key writes: every key lands (nest)
Promise.all of disjoint-key writes: every key lands (nuxt)
Promise.all of disjoint-key writes: every key lands (tanstack-start)
setAttributesInsideStepWorkflow: step-body calls append attributed native events (nextjs-turbopack)
setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly (vite)
sleepWinsRaceWorkflow (vite)
start: initial attributes are seeded on run creation (example)
start: initial attributes are seeded on run creation (nextjs-turbopack)
validation DX: invalid writes throw catchable FatalErrors naming rule and limit (express)
validation DX: invalid writes throw catchable FatalErrors naming rule and limit (fastify)
validation DX: invalid writes throw catchable FatalErrors naming rule and limit (nitro)
validation DX: invalid writes throw catchable FatalErrors naming rule and limit (tanstack-start)
workflow throws after awaited setAttributes: attribute still persists on the failed run (express)
workflow throws after awaited setAttributes: attribute still persists on the failed run (fastify)
workflow throws after awaited setAttributes: attribute still persists on the failed run (nextjs-turbopack)
workflow throws after awaited setAttributes: attribute still persists on the failed run (nextjs-webpack)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

cold-start-warmup · suite warmup (tanstack-start) · at 18:35:21Z · abandoned wrun_01M0JSNG5YY2EJ38F5CWJ1C0R0
run-pickup-stall · hookCleanupTestWorkflow - hook token reuse after workflow completion (nextjs-webpack) · at 18:41:40Z · abandoned wrun_01M0JT1EKJHZM46BTW6J4K101P

E2E Test Summary

Summary

	Passed	Failed	Skipped	Total
❌ ▲ Vercel Production	3337	241	742	4320
✅ 💻 Local Development	3922	0	558	4480
✅ 📦 Local Production	3922	0	558	4480
✅ 🐘 Local Postgres	3922	0	558	4480
✅ 🪟 Windows	320	0	0	320
✅ 🌐 Cross-language Conformance	9	0	132	141
✅ vercel-multi-region	27	0	0	27
Total	15459	241	2548	18248

Details by Category

❌ ▲ Vercel Production

App	Passed	Failed	Skipped
❌ astro-node	121	11	28
❌ astro-quickjs	123	9	28
❌ example-node	123	9	28
❌ example-quickjs	121	11	28
❌ express-node	123	9	28
❌ express-quickjs	120	12	28
❌ fastify-node	122	10	28
❌ fastify-quickjs	122	10	28
❌ hono-node	125	7	28
❌ hono-quickjs	123	9	28
❌ nest-node	123	9	28
❌ nest-quickjs	121	11	28
❌ nextjs-turbopack-node	152	5	3
❌ nextjs-turbopack-quickjs	148	9	3
❌ nextjs-webpack-node	148	9	3
❌ nextjs-webpack-quickjs	146	11	3
❌ nitro-node	125	7	28
❌ nitro-quickjs	121	11	28
❌ nuxt-node	124	8	28
❌ nuxt-quickjs	121	11	28
✅ python-node	8	0	152
❌ sveltekit-node	146	5	9
❌ sveltekit-quickjs	140	11	9
❌ tanstack-start-node	124	8	28
❌ tanstack-start-quickjs	122	10	28
❌ vite-node	124	8	28
❌ vite-quickjs	121	11	28

✅ 💻 Local Development

App	Passed	Skipped
✅ astro-stable-node	134	26
✅ astro-stable-quickjs	134	26
✅ express-stable-node	134	26
✅ express-stable-quickjs	134	26
✅ fastify-stable-node	134	26
✅ fastify-stable-quickjs	134	26
✅ hono-stable-node	134	26
✅ hono-stable-quickjs	134	26
✅ nest-stable-node	134	26
✅ nest-stable-quickjs	134	26
✅ nextjs-turbopack-canary-node	141	19
✅ nextjs-turbopack-canary-quickjs	141	19
✅ nextjs-turbopack-stable-node	160	0
✅ nextjs-turbopack-stable-quickjs	160	0
✅ nextjs-webpack-canary-node	141	19
✅ nextjs-webpack-canary-quickjs	141	19
✅ nextjs-webpack-stable-node	160	0
✅ nextjs-webpack-stable-quickjs	160	0
✅ nitro-stable-node	134	26
✅ nitro-stable-quickjs	134	26
✅ nuxt-stable-node	134	26
✅ nuxt-stable-quickjs	134	26
✅ sveltekit-stable-node	153	7
✅ sveltekit-stable-quickjs	153	7
✅ tanstack-start-node	134	26
✅ tanstack-start-quickjs	134	26
✅ vite-stable-node	134	26
✅ vite-stable-quickjs	134	26

✅ 📦 Local Production

App	Passed	Skipped
✅ astro-stable-node	134	26
✅ astro-stable-quickjs	134	26
✅ express-stable-node	134	26
✅ express-stable-quickjs	134	26
✅ fastify-stable-node	134	26
✅ fastify-stable-quickjs	134	26
✅ hono-stable-node	134	26
✅ hono-stable-quickjs	134	26
✅ nest-stable-node	134	26
✅ nest-stable-quickjs	134	26
✅ nextjs-turbopack-canary-node	141	19
✅ nextjs-turbopack-canary-quickjs	141	19
✅ nextjs-turbopack-stable-node	160	0
✅ nextjs-turbopack-stable-quickjs	160	0
✅ nextjs-webpack-canary-node	141	19
✅ nextjs-webpack-canary-quickjs	141	19
✅ nextjs-webpack-stable-node	160	0
✅ nextjs-webpack-stable-quickjs	160	0
✅ nitro-stable-node	134	26
✅ nitro-stable-quickjs	134	26
✅ nuxt-stable-node	134	26
✅ nuxt-stable-quickjs	134	26
✅ sveltekit-stable-node	153	7
✅ sveltekit-stable-quickjs	153	7
✅ tanstack-start-node	134	26
✅ tanstack-start-quickjs	134	26
✅ vite-stable-node	134	26
✅ vite-stable-quickjs	134	26

✅ 🐘 Local Postgres

App	Passed	Skipped
✅ astro-stable-node	134	26
✅ astro-stable-quickjs	134	26
✅ express-stable-node	134	26
✅ express-stable-quickjs	134	26
✅ fastify-stable-node	134	26
✅ fastify-stable-quickjs	134	26
✅ hono-stable-node	134	26
✅ hono-stable-quickjs	134	26
✅ nest-stable-node	134	26
✅ nest-stable-quickjs	134	26
✅ nextjs-turbopack-canary-node	141	19
✅ nextjs-turbopack-canary-quickjs	141	19
✅ nextjs-turbopack-stable-node	160	0
✅ nextjs-turbopack-stable-quickjs	160	0
✅ nextjs-webpack-canary-node	141	19
✅ nextjs-webpack-canary-quickjs	141	19
✅ nextjs-webpack-stable-node	160	0
✅ nextjs-webpack-stable-quickjs	160	0
✅ nitro-stable-node	134	26
✅ nitro-stable-quickjs	134	26
✅ nuxt-stable-node	134	26
✅ nuxt-stable-quickjs	134	26
✅ sveltekit-stable-node	153	7
✅ sveltekit-stable-quickjs	153	7
✅ tanstack-start-node	134	26
✅ tanstack-start-quickjs	134	26
✅ vite-stable-node	134	26
✅ vite-stable-quickjs	134	26

✅ 🪟 Windows

App	Passed	Failed	Skipped
✅ nextjs-turbopack-node	160	0	0
✅ nextjs-turbopack-quickjs	160	0	0

✅ 🌐 Cross-language Conformance

App	Passed	Failed	Skipped
✅ python	9	0	132

✅ vercel-multi-region

App	Passed	Failed	Skipped
✅ nextjs-turbopack	27	0	0

📋 View full workflow run

github-actions · 2026-08-20T20:42:03Z

📊 Workflow Benchmarks

❌ The benchmark run for 8e045a6 failed. See the run logs for details.

No benchmark results were produced.

ℹ️ Metric definitions & methodology

_{Metrics —}

_{All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).}

_{Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.}

github-actions · 2026-08-20T20:45:23Z

Sim World

Simulated world deterministic testing for races. Traces

🟠 world-sim scenario book — 1 fail of 41 total

fence=per-spec

scenario	outcome	events	virt	replay
✅ `smoke-no-steps`	completed	3	0ms	ok
✅ `smoke-one-step`	completed	6	0ms	ok
✅ `hook-at-step-started`	completed	12	0ms	ok
✅ `hook-at-step-completed`	completed	12	0ms	ok
✅ `hook-at-hook-created`	completed	12	0ms	ok
✅ `deadline-hook-wins`	completed	7	1.0h	ok
✅ `deadline-expires`	completed	7	1.0h	ok
✅ `long-sleep`	completed	11	30.0d	ok
✅ `hook-never-arrives`	stalled	3	0ms	skipped
✅ `step-retries-twice`	completed	10	2.0s	ok
✅ `parallel-steps`	completed	9	0ms	ok
✅ `hook-on-execution-state`	completed	12	0ms	ok
✅ `peek-hook-before-branch`	completed	12	0ms	ok
✅ `peek-hook-after-branch`	completed	12	0ms	ok
✅ `peek-hook-at-registration`	completed	12	0ms	ok
✅ `race-hook-before-probe`	completed	12	0ms	ok
✅ `race-hook-after-probe`	completed	12	0ms	ok
✅ `race-duplicate-delivery`	completed	13	0ms	ok
✅ `attr-hook-before-step`	completed	11	0ms	ok
✅ `attr-hook-after-step`	completed	11	0ms	ok
✅ `attr-from-step-body`	completed	13	0ms	ok
✅ `fork-hook-after-timeout`	completed	14	1.0m	ok
✅ `fork-hook-before-timeout`	completed	14	1.0m	ok
✅ `count-hook-after-timeout`	completed	17	1.0m	ok
✅ `count-hook-before-timeout`	completed	20	1.0m	ok
✅ `stale-read-step-count-fork`	completed	20	1.0m	ok
✅ `stale-read-equal-step-counts`	completed	14	1.0m	ok
✅ `step-vs-step-fork`	completed	12	0ms	ok
✅ `step-vs-step-fork-fenced`	completed	12	0ms	ok
✅ `fence-catches-benign-direction`	completed	12	5ms	ok
✅ `in-flight-before-decision`	completed	17	1.0m	ok
❌ `in-flight-before-decision-counted`	completed	17	1.0m	ok
✅ `in-flight-after-decision`	completed	19	2.0m	ok
✅ `stale-read-step-count-fork-fenced`	completed	20	1.0m	ok
✅ `fork-hook-wins`	completed	13	1.0m	ok
✅ `fork-timeout-wins`	completed	13	1.0m	ok
✅ `unclaimed-payload-under-fork`	completed	17	1.0m	ok
✅ `claimed-payload-under-fork`	completed	17	1.0m	ok
✅ `writers-independent-step-bodies`	completed	12	0ms	ok
✅ `writers-scripted-tempo`	completed	12	0ms	ok
✅ `cancel-mid-step`	cancelled	7	0ms	skipped

Full trace: world-sim.txt

github-actions · 2026-08-20T20:53:03Z

Event Log Race Repro

vercel clean, 20 runs
local clean, 20 runs
postgres clean, 20 runs

Run History

Run	Lane	Total	Complete	Corrupt	Stuck	Other
08-20 22:40	vercel	20	20	0	0	0
	local	20	20	0	0	0
	postgres	20	20	0	0	0
08-21 16:46	vercel	–	–	–	–	–
	local	–	–	–	–	–
	postgres	–	–	–	–	–
08-21 16:56	vercel	20	20	0	0	0
	local	20	20	0	0	0
	postgres	20	20	0	0	0
08-21 17:56	vercel	20	20	0	0	0
	local	20	20	0	0	0
	postgres	20	20	0	0	0
08-21 18:40	vercel	20	20	0	0	0
	local	20	20	0	0	0
	postgres	20	20	0	0	0

Config

20 runs / step-storm 6, hook-storm 6, blocked-branch 6, hook-sleep 2 / c8 / 6x8 / watchdog 2500ms / step 2200±250ms / stagger 400ms / burst 4000+1200ms / poke 750ms / poke max 64 / timeout 240000ms

A step delivery's log-order turnstile waits on lower-index ARMED barriers, and an armed wait can be parked behind an unclaimed buffered hook payload that only the idle-gated safety net can retire. Termination there rests on the spinning delivery not counting as a parked committed delivery (resolvesOnItsOwn excludes it), keeping the net's gate reachable. Pin that shape with a replay in both flag states so a turnstile or idle-predicate change that deadlocks it fails fast instead of only hanging e2e lanes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

vercel

Additional Suggestion:

The generated e2e diagnostics sidecar glob (e2e-diagnostics-*.json) is not gitignored, so per-run harness output written to the repo root keeps getting accidentally committed.

…son`) is not gitignored, so per-run harness output written to the repo root keeps getting accidentally committed. This commit fixes the issue reported at .gitignore:53 ## Status: partially fixed, gitignore rule still needed The stray artifact `e2e-diagnostics-express-local.json` from this PR was **removed** by the follow-up commit `fd6e18a` ("Remove stray local e2e diagnostics artifact"). That part of the original suggestion is now done. However, the **root cause remains unaddressed**: there is still no `.gitignore` rule for the sidecar pattern, so the harness will re-stage these files on the next e2e run. ### Evidence - `git ls-files | grep e2e-diagnostics` no longer lists `e2e-diagnostics-express-local.json` (removed), but **still** lists a pre-existing sidecar `e2e-diagnostics-nextjs-turbopack-vercel.json` — proof this is a recurring accidental-commit pattern. - `git check-ignore e2e-diagnostics-express-local.json` returned exit `1` (no matching pattern) before this change. - `writeDiagnosticsSidecar()` in `packages/core/e2e/utils.ts` writes `e2e-diagnostics-<APP_NAME>-<backend>.json` to `process.cwd()` during e2e runs — clearly per-run output, not source. ### Fix Add an ignore rule to `.gitignore` (file-removal is already handled by `fd6e18a`): ``` # Per-run e2e diagnostics sidecars written to the repo root by the harness # (writeDiagnosticsSidecar in packages/core/e2e/utils.ts) e2e-diagnostics-*.json ``` After the change, `git check-ignore e2e-diagnostics-express-local.json` matches (exit `0`), so future runs won't re-stage the sidecar. The already-tracked `e2e-diagnostics-nextjs-turbopack-vercel.json` is left in place (out of scope), though the new rule covers that pattern for any future untracked instances. Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com> Co-authored-by: VaguelySerious <mittgfu@gmail.com>

TooTallNate

AI: Request changes. The new runtime-tuning documentation points users to the wrong VM selector. I found no additional blocking code issues.

TooTallNate · 2026-08-21T18:15:30Z

+- Experimental. Pins correlation-ID draw order to event-log order: a branch-deciding delivery (a step result, hook payload, or wait completion) resolves to the workflow only after every earlier-in-log delivery's continuation has fully quiesced, and never ahead of a lower-slot delivery that is committed to happening.
+- Without it, a delivery that resolves while an earlier delivery's continuation is still a few microtask hops from its next step/hook/wait call can overtake it on the run's shared correlation-ID sequence. Draw order — and therefore correlation IDs — then depends on how much of the event log a replay had loaded, and two concurrent replays holding different-length prefixes can bind one ID to two different entities, failing the run with `CORRUPTED_EVENT_LOG`.
+- Costs one event-loop turn (roughly 15-20 microseconds via `setImmediate`) per branch-deciding delivery during replay, more when continuations genuinely overlap. Measured on a 100-step sequential replay: about 2ms added end to end.
+- Only applies to the default Node.js VM engine. `WORKFLOW_RUNTIME=quickjs` has its own event feed and correlation-ID sequence and is unaffected by this setting.


AI: The engine selector is WORKFLOW_VM, as documented below, not WORKFLOW_RUNTIME. As written, users following this guidance will remain on the Node VM while believing they selected QuickJS. Please change this to WORKFLOW_VM=quickjs.

(AI) Right — fixed in 8e045a6: the line now reads WORKFLOW_VM=quickjs, matching the selector documented in the WORKFLOW_VM section below. Verified WORKFLOW_RUNTIME appears nowhere else in docs or code; my earlier reply on the quickjs-scope thread used the wrong name too, same correction applies there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

github-actions · 2026-08-21T18:32:14Z

Backport to stable failed for 9b1b8c7 due to a workflow error (backport job run).

This is usually an infrastructure problem (e.g. the configured AI model could not be found, an AI Gateway error, or an opencode crash) rather than a merge conflict. Check the job logs linked above for details.

Once the underlying issue is fixed, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

9b1b8c711104fd507327aafc8cb965738f315e29

github-actions · 2026-08-21T19:48:46Z

Backport to stable failed for 9b1b8c7 due to a workflow error (backport job run).

This is usually an infrastructure problem (e.g. the configured AI model could not be found, an AI Gateway error, or an opencode crash) rather than a merge conflict. Check the job logs linked above for details.

Once the underlying issue is fixed, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

9b1b8c711104fd507327aafc8cb965738f315e29

vercel Bot deployed to Preview – workbench-python-workflow August 20, 2026 20:41 View deployment

VaguelySerious added the event-log-race-repro Run the event log race reproduction job label Aug 20, 2026

vercel Bot deployed to Preview – workflow-web August 20, 2026 20:42 View deployment

vercel Bot deployed to Preview – workflow-tarballs August 20, 2026 20:42 View deployment

vercel Bot deployed to Preview – workbench-astro-workflow August 20, 2026 20:42 View deployment

vercel Bot deployed to Preview – workbench-hono-workflow August 20, 2026 20:42 View deployment

vercel Bot deployed to Preview – workbench-tanstack-start-workflow August 20, 2026 20:42 View deployment

vercel Bot deployed to Preview – example-workflow August 20, 2026 20:43 View deployment

vercel Bot deployed to Preview – workbench-fastify-workflow August 20, 2026 20:43 View deployment

vercel Bot deployed to Preview – workbench-express-workflow August 20, 2026 20:43 View deployment

vercel Bot deployed to Preview – workbench-nitro-workflow August 20, 2026 20:43 View deployment

vercel Bot deployed to Preview – workbench-nuxt-workflow August 20, 2026 20:43 View deployment

vercel Bot deployed to Preview – workbench-nestjs-workflow August 20, 2026 20:43 View deployment

vercel Bot deployed to Preview – workbench-vite-workflow August 20, 2026 20:43 View deployment

vercel Bot deployed to Preview – workbench-sveltekit-workflow August 20, 2026 20:43 View deployment

vercel Bot deployed to Preview – example-nextjs-workflow-webpack August 20, 2026 20:43 View deployment

vercel Bot deployed to Preview – example-nextjs-workflow-turbopack August 20, 2026 20:43 View deployment

vercel Bot deployed to Preview – workflow-swc-playground August 20, 2026 20:44 View deployment

vercel Bot deployed to Preview – workflow-docs August 20, 2026 20:45 View deployment

vercel Bot deployed to Preview – workbench-python-workflow August 20, 2026 22:31 View deployment

vercel Bot deployed to Preview – workflow-web August 20, 2026 22:32 View deployment

vercel Bot deployed to Preview – workflow-tarballs August 20, 2026 22:32 View deployment

vercel Bot deployed to Preview – example-nextjs-workflow-turbopack August 21, 2026 16:46 View deployment

vercel Bot deployed to Preview – workbench-nuxt-workflow August 21, 2026 16:47 View deployment

vercel Bot deployed to Preview – example-nextjs-workflow-webpack August 21, 2026 16:47 View deployment

vercel Bot deployed to Preview – workflow-docs August 21, 2026 16:47 View deployment

vercel Bot deployed to Preview – workflow-swc-playground August 21, 2026 16:49 View deployment

vercel Bot reviewed Aug 21, 2026

View reviewed changes

vercel Bot deployed to Preview – workbench-python-workflow August 21, 2026 17:49 View deployment

vercel Bot deployed to Preview – workflow-web August 21, 2026 17:49 View deployment

vercel Bot deployed to Preview – example-nextjs-workflow-turbopack August 21, 2026 17:49 View deployment

vercel Bot deployed to Preview – workbench-hono-workflow August 21, 2026 17:49 View deployment

vercel Bot deployed to Preview – workbench-fastify-workflow August 21, 2026 17:50 View deployment

vercel Bot deployed to Preview – workbench-express-workflow August 21, 2026 17:50 View deployment

vercel Bot deployed to Preview – workbench-nestjs-workflow August 21, 2026 17:50 View deployment

vercel Bot deployed to Preview – workflow-tarballs August 21, 2026 17:50 View deployment

vercel Bot deployed to Preview – workbench-nitro-workflow August 21, 2026 17:50 View deployment

vercel Bot deployed to Preview – workbench-tanstack-start-workflow August 21, 2026 17:50 View deployment

TooTallNate previously requested changes Aug 21, 2026

View reviewed changes

pranaygp approved these changes Aug 21, 2026

View reviewed changes

[docs] Fix engine selector name: WORKFLOW_VM, not WORKFLOW_RUNTIME

8e045a6

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

github-actions Bot mentioned this pull request Aug 21, 2026

Version Packages (beta) #3694

Closed

github-actions Bot mentioned this pull request Aug 21, 2026

Version Packages (beta) #3717

Open

This was referenced Aug 21, 2026

Version Packages (beta) ben-vargas/workflow#104

Open

Version Packages (beta) Jokacar10/jkworkflow#519

Open

Version Packages (beta) Thegreatsura/workflow#1

Open

Navigation Menu

Uh oh!

[core] Pin correlation-id draw order to event-log order - #3700

[core] Pin correlation-id draw order to event-log order#3700
VaguelySerious merged 6 commits into
mainfrom
peter/log-order-draws

VaguelySerious commented Aug 20, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

vercel Bot left a comment

Uh oh!

TooTallNate left a comment

Uh oh!

TooTallNate Aug 21, 2026

Uh oh!

VaguelySerious Aug 21, 2026

Uh oh!

github-actions Bot commented Aug 21, 2026

Uh oh!

github-actions Bot commented Aug 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

VaguelySerious commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

What this changes

Tests

Costs and open questions

Relation to #3179

Uh oh!

changeset-bot Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

vercel Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 E2E Test Results

❌ Failed E2E Tests

⚠️ Flaky E2E Tests (passed on retry)

🛠 Infra Events (absorbed by the harness)

E2E Test Summary

Uh oh!

github-actions Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Workflow Benchmarks

Uh oh!

github-actions Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Sim World

Uh oh!

github-actions Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Event Log Race Repro

Run History

Uh oh!

vercel Bot left a comment

Choose a reason for hiding this comment

Uh oh!

TooTallNate left a comment

Choose a reason for hiding this comment

Uh oh!

TooTallNate Aug 21, 2026

Choose a reason for hiding this comment

Uh oh!

VaguelySerious Aug 21, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 21, 2026

Uh oh!

github-actions Bot commented Aug 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

VaguelySerious commented Aug 20, 2026 •

edited

Loading

changeset-bot Bot commented Aug 20, 2026 •

edited

Loading

vercel Bot commented Aug 20, 2026 •

edited

Loading

github-actions Bot commented Aug 20, 2026 •

edited

Loading

github-actions Bot commented Aug 20, 2026 •

edited

Loading

github-actions Bot commented Aug 20, 2026 •

edited

Loading

github-actions Bot commented Aug 20, 2026 •

edited

Loading