perf: reduce publish duplicate and coverage reads

[guangyu-reflexio]

Summary

Reduce publish acknowledgement latency by eliminating duplicate preflight reads and sharing post-commit coverage work. Duplicate request IDs still reject the entire admission transaction, and optional reporting remains best-effort after durable commit.

Changes

• Introduce the atomic add_request_if_absent storage contract and SQLite implementation; preserve existing add_request upsert behavior. Custom backends must implement the atomic contract.
• Replace both publish duplicate reads with one atomic insert-and-reject operation. External receipt replay still happens before embedding preparation; ordinary duplicates can prepare embeddings before rejection.
• Read extraction status and counts through one connection and admission lookup, select explicit cursor columns, and keyset-page completed window effects.
• Add connection-attempt timing counters and regression coverage for rollback, replay, reporting failures, status equivalence, and receipt pagination.
• Update affected test doubles and deterministic fixtures to match the storage and scheduling contracts.

Test Plan

• OSS unit/integration suites exercised: 6,346 passed in the initial run; the one stale mock was fixed, and all 205 tests in the affected final selection passed.
• OSS e2e: 48 passed; 43 low-priority skips and 44 credential-dependent tests deselected.
• Plugin suite: 183 passed, 2 backend-unavailable skips.
• Ruff passed. Pyright reports no errors; the standalone plugin test has an optional dependency import warning.
• Atomic duplicate and concurrent HTTP publication behavior also verified against disposable PostgreSQL with deterministic embedding/auth boundaries.
• TestSprite CLI is unavailable. Production latency qualification remains outstanding; local loopback timings do not establish the 5,000 ms production target.

Summary by CodeRabbit

• Bug Fixes

  • Duplicate publishes are rejected without replacing the existing request.
  • A publish remains successful if post-commit status and count reporting fails; unavailable reporting fields are omitted.

• Improvements

  • Extraction status and output counts are reported together, with consistent results across larger reports.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-25T22:05:53.714882Z	4201ed4	PR opened
🔒 Security Review	✅ Completed 2026-09-25T22:08:07.148833Z	4201ed4	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: ReflexioAI/reflexio/.coderabbit.yaml

Review profile: CHILL

Plan: Essentials

Run ID: 3c4e08d4-3836-4def-96c4-013fd641e93b

📥 Commits

Reviewing files that changed from the base of the PR and between 5fa91c2 and 4201ed4.

📒 Files selected for processing (23)

• developer.md
• reflexio/integrations/openclaw/plugin/tests/test_reflexio_adapter.py
• reflexio/lib/_interactions.py
• reflexio/server/publish_timing.py
• reflexio/server/services/generation_service.py
• reflexio/server/services/storage/retention_sweep.py
• reflexio/server/services/storage/sqlite_storage/_requests.py
• reflexio/server/services/storage/storage_base/_extraction_stream.py
• reflexio/server/services/storage/storage_base/_requests.py
• tests/lib/test_interactions_unit.py
• tests/lib/test_profile_workflows_unit.py
• tests/server/cache/test_reflexio_cache.py
• tests/server/llm/test_token_accounting.py
• tests/server/services/durable_learning/test_window_codec_compat.py
• tests/server/services/durable_learning/test_worker_failure_locator.py
• tests/server/services/lineage/test_gc_scheduler_retention_class_c.py
• tests/server/services/storage/test_retention_sweep.py
• tests/server/services/storage/test_storage_contract_commit_scope.py
• tests/server/services/storage/test_storage_contract_extraction_stream.py
• tests/server/services/test_generation_service_durable_enqueue.py
• tests/server/services/test_generation_service_publish_timing.py
• tests/server/services/test_generation_service_scheduling.py
• tests/server/services/test_publish_unattributed_phases.py

Included review availability: This review used your included allowance. Your plan provides up to 5 included reviews per hour; 1 remain after this review. Your free on-demand review promotion remains active until October 9, 2026 at 6:00 PM UTC.

---

📝 Walkthrough

Walkthrough

Publishing now uses an atomic request insert inside the commit scope. Extraction status and counts are returned through a combined post-commit report. The report uses one storage connection and paginates completed extraction windows.

Changes

Publishing and storage

Layer / File(s)	Summary
Atomic publish admission reflexio/server/services/generation_service.py, reflexio/server/services/storage/storage_base/_requests.py, reflexio/server/services/storage/sqlite_storage/_requests.py, reflexio/server/publish_timing.py, tests/server/services/*, tests/server/services/storage/test_storage_contract_commit_scope.py	The publish path replaces the duplicate read and request write with add_request_if_absent inside commit_scope. SQLite inserts without replacing an existing request; add_request retains its upsert behavior. The tests cover duplicate rejection, rollback, and publish timing.
Combined extraction reporting developer.md, reflexio/lib/_interactions.py, reflexio/server/services/storage/storage_base/_extraction_stream.py, tests/lib/*, tests/server/services/test_generation_service_publish_timing.py, tests/server/services/test_publish_unattributed_phases.py, tests/server/services/storage/test_storage_contract_extraction_stream.py	extraction_report returns status and counts after admission is read once. Count calculation processes completed windows in batches of up to 200. Tests cover existing status results, query use, pagination, and reporting failures.
Adjacent test and terminology updates reflexio/integrations/openclaw/plugin/tests/test_reflexio_adapter.py, reflexio/server/services/storage/retention_sweep.py, tests/server/cache/test_reflexio_cache.py, tests/server/llm/test_token_accounting.py, tests/server/services/durable_learning/*, tests/server/services/lineage/test_gc_scheduler_retention_class_c.py, tests/server/services/storage/test_retention_sweep.py	Tests isolate environment variables, add explicit assertions, and use an autospecced request context. Comments and test descriptions update terminology.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Refactor

Sequence Diagram(s)

sequenceDiagram
  participant GenerationService
  participant SQLiteRequestMixin
  participant SQLiteDatabase
  GenerationService->>SQLiteRequestMixin: add_request_if_absent within commit_scope
  SQLiteRequestMixin->>SQLiteDatabase: INSERT with ON CONFLICT DO NOTHING
  SQLiteDatabase-->>SQLiteRequestMixin: whether a row was inserted
  SQLiteRequestMixin-->>GenerationService: insertion result

Loading

Suggested reviewers: yyiilluu

Merge Risk: ⚪ Minimal · up to 4201e

The shipped publish workflow has no identified issue blocking merge. Operators using external storage backends should confirm they have implemented the new storage contract.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.94% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 53 functions across 22 files. (1 skipped:…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main changes: reducing duplicate-request checks and combining coverage reads during publishing.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 50.94% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 53 functions across 22 files. (1 skipped: 1 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

_{Comment @coderabbitai help to get the list of available commands.}