fix(libsy): record decision call lifecycle telemetry

[nachiketb-nvidia]

What

Record Decision Model call counts, duration, outcome, and available response metadata in the libsy driver, including failed and abandoned calls.

Why

CallDecision currently emits no call metrics and leaves its span without an outcome or usage. Recording only in an HTTP host misses custom hosts and calls dropped during cancellation.

Addresses Greg's request for record and Drop on the decision step.

How

• Add started, a retained span, and reply ownership to CallDecision. Reply, failure, and unfulfilled drop each record exactly once.
• Emit switchyard.decision_calls and switchyard.decision_call_duration_ms, including host queueing, with algorithm, selected model, and outcome labels.
• Record outcome, available token counts, and provider response/model IDs on libsy.decision_call. Keep request content, answers, and raw errors out of the span.
• Document the new telemetry. Reuse the existing token-field helper for LLM and decision spans.

Notes for reviewers

Independent of #907: based on main; either PR can merge first. Provider HTTP-attempt metrics and client error detail are outside this driver change.

Start with CallDecision, then the observability helpers. Production Rust is +99/-16 (net +83).

Validation

• One new test first failed on main with a missing decision-call counter; it now passes for reply, provider error, host failure, dropped call, and cancelled run. It checks one count/duration sample, host waiting time, span fields, omitted unknown usage, and content exclusion.
• 34 focused tests passed: driver (15), observability integration (18), and decision capability routing (1).
• Affected-crate Clippy with warnings denied, strict Rustdoc, formatting, diff checks, and strict MkDocs build passed.
• No full test suite or live provider calls.

Summary by CodeRabbit

• Observability
  • Decision Model calls now report counts and durations for replies, failures, and calls that end without a reply, including time spent queued by the host.
  • Successful replies include available response, model, and token-usage details in tracing data. Request content, answers, and error details are not recorded.
• Documentation
  • Updated the OpenTelemetry reference to describe Decision Model call metrics, span attributes, and outcomes.

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-908/
Built to branch gh-pages at 2026-10-02 21:58 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: NVIDIA-NeMo/Switchyard/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 91daf8b1-2f5b-4e29-aab2-40dc66cfb4c6

📥 Commits

Reviewing files that changed from the base of the PR and between 84075f1 and b63c7c1.

📒 Files selected for processing (4)

• crates/libsy-llm-client/tests/observability.rs
• crates/libsy/src/core/algorithm.rs
• crates/libsy/src/observability.rs
• docs/reference/opentelemetry.md

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

---

Walkthrough

Decision calls now record a terminal outcome and duration. Successful replies add available response and token fields to the originating span. Tests cover replies, errors, explicit failures, dropped calls, and cancellation. The OpenTelemetry reference describes the spans, metrics, and timing behavior.

Changes

Decision-call observability

Layer / File(s)	Summary
Decision-call lifecycle instrumentation crates/libsy/src/core/algorithm.rs	Driver::call_decision timestamps each call and attaches its current span. Answered, failed, and unfulfilled calls record terminal telemetry. Successful responses add response telemetry to the originating span.
Telemetry recording and validation crates/libsy/src/observability.rs, crates/libsy-llm-client/tests/observability.rs, docs/reference/opentelemetry.md	Decision-call counters and duration samples include algorithm, selected model, and outcome. Tests verify terminal paths, span fields, and that decision calls do not increment llm_calls. The reference documents recorded fields, host queueing, and outcomes for drops and cancellation.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to b63c7

Decision-call telemetry covers the inspected reply, failure, and cancellation paths. No issue remains that should block merging after normal checks.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 3 files. (1 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: recording decision-call lifecycle telemetry.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 3 files. (1 skipped: 1 unsupported.)

• Fix all pre-merge checks with AI

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

---

_{A rabbit checks each call’s reply,
And notes the time as moments fly.
A dropped call leaves an error trace,
A good reply adds tokens in place.
The spans record what calls have done,
Then close their books when work is done.}

_{Comment @coderabbitai help to get the list of available commands.}