feat(replication): add first-audit scheduler observability

[dirvine]

Summary

Adds observability to ADR-0004's deterministic monetised-pin first-audit scheduler without changing scheduling or protocol semantics.

• labels first-audit events with audit_trigger=first_monetized;
• records stable scheduling/terminal outcomes (queued, coalesced, cooldown_deferred, outside_answerability_window, launched, passed, timeout, failed);
• includes peer/pin/key-count, pending, inflight and elapsed context where available;
• emits a cumulative five-minute scheduler summary suitable for Elasticsearch;
• adds a focused test for terminal outcome classification and stable labels.

This is the common observability layer for the bounded first-audit scheduler work. It deliberately does not change queue capacity, retry policy, cooldowns, launch rates, payment verification, trust handling, audit mechanics or commitment semantics.

Why

A matched DEV-01/DEV-02 ablation found that detaching the monetised-pin sender reduced:

• generic one-key timeout failures by 99.955%;
• storage-commitment audit traffic by 97.7%;
• responsible-audit send failures by 65.5%;

while preserving 100% upload/download success and improving transfer-tail duration and node resource load. The current logs cannot provide a complete eligible → queued/coalesced → launched → terminal first-audit funnel. This PR supplies that denominator before the scheduler is redesigned.

Verification

• cargo fmt --all -- --check
• cargo check --all-targets --all-features
• cargo test first_audit_terminal_outcomes_are_stable --all-features
• cargo test monetized --all-features
• cargo test --lib --all-features — 684 passed
• cargo clippy --all-targets --all-features -- -D warnings -D clippy::panic -D clippy::unwrap_used -D clippy::expect_used

Review notes

The per-event records are DEBUG; only the five-minute cumulative summary is INFO, to avoid making normal fleet logging materially noisier.

[Copilot]

Pull request overview

Adds an observability layer to the ADR-0004 deterministic “first-audit” scheduler for monetized pins, emitting per-event debug logs and a low-frequency cumulative summary to support fleet-level funnel analysis without changing scheduling/protocol behavior.

Changes:

• Introduces atomic counters and stable terminal outcome classification for first-audit runs.
• Emits per-event DEBUG logs for queueing/coalescing/launching/terminal outcomes and a 5-minute INFO summary.
• Adds a unit test to ensure terminal outcome classification and string labels remain stable.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
src/replication/mod.rs	Adds first-audit scheduler observability (counters, logs, terminal outcome classification) and a stability unit test.
src/replication/config.rs	Adds the 5-minute summary interval constant for the new observability summary.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.