Skip to content

Stabilize live plugin validation - #24

Merged
dimavrem22 merged 6 commits into
mainfrom
fix/live-ci-stability
Aug 8, 2026
Merged

Stabilize live plugin validation#24
dimavrem22 merged 6 commits into
mainfrom
fix/live-ci-stability

Conversation

@dimavrem22

@dimavrem22 dimavrem22 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Executive Summary

This PR makes complete live plugin validation strict, repeatable, and non-blocking for merges.

  • Correlates each current call to one record per owner and reads transcripts from the authoritative owner.
  • Makes A2A readiness, mock responses, external-event handoff, and hosted post-call SMS settlement deterministic.
  • Preserves exact two-way speech, call mode, action, recipient, accepted-message, and single-delivery behavior.

Description

The live voice harness now sweeps matching active calls, snapshots both owners, applies a request-time lower bound, requires one stable current record per owner, and cleans up every fresh matching record. It checks driver-local speech only on the driver record and requires two-way conversation plus agent-local speech on the AUT record. Hosted scenarios use a speech-safe three-word marker while retaining exact action, target, and single accepted-message assertions.

Hosted post-call turns use a default-deny tool map and a durable SMS journal. Successful sends retain their accepted message ID so later delivery-failure events cannot escape into an unrelated recovery turn and create a duplicate. Pre-delivery policy audit rows remain visible in diagnostics but do not count as accepted SMS messages. A2A receivers are enabled and verified before each scenario, mock completions receive request-unique response IDs, external events acknowledge after durable handoff, setup-only network operations use bounded retries, and public failure output contains only state and counts.

Reason

Live failures were recurring across mutable receiver state, stale reply/call correlation, incorrect transcript ownership, reused mock response IDs, long-running external acknowledgements, model-sensitive hosted actions, and duplicate hosted delivery recovery. The fixes address those failure modes without retrying calls or messages or weakening end-to-end assertions.

Decisions

  • Call correlation: Use strict plugin-side snapshots, request-time bounds, directions, numbers, timestamps, and stability because the existing records provide sufficient ownership evidence.
  • Hosted settlement: Persist the exact accepted SMS message ID and suppress only its generic delivery recovery so unrelated message failures retain normal behavior.
  • Test oracle: Count every fresh accepted marker message while excluding only pre-delivery policy audit rows, which never reached delivery.
  • Retry boundary: Retry only idempotent setup and read operations; outbound messages, call placement, and behavioral attempts remain single-shot.
  • Merge gating: Keep unit, current-host contract, and loader checks required while the complete live matrix continues to run as non-blocking validation.

Testing

  • npm run lint — passed across 152 files.
  • npm run typecheck and npm run build — passed.
  • npm test — 70 files and 827 tests passed.
  • python3 -m unittest tests/live/test_a2a_preflight.py — 3 tests passed.
  • Python helper compilation, shell syntax, workflow YAML parsing, and git diff --check — passed.
  • Complete live cycle 1 — channels, A2A, voice, external events, and aggregate passed.
  • Complete live cycle 2 — the same full matrix passed on the unchanged commit.
  • Complete live cycle 3 — the same full matrix passed on the unchanged commit for a third consecutive cycle.
  • Fast checks attempts 1–3 — unit, current-host contract, and loader smoke passed in all three attempts.

@dimavrem22
dimavrem22 merged commit 7de97ee into main Aug 8, 2026
45 checks passed
@dimavrem22
dimavrem22 deleted the fix/live-ci-stability branch August 8, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant