feat(yang-push): migrate to yang-rs-5 / libyang 5.4.9, revamp OTEL metrics, refresh example YANG modules

[rodonile]

Summary

This PR contains the following commits:

1. feat(yang-push): update validation actor to yang-rs-5 (libyang 5.4.9) — upgrading the YANG-Push validation actor to the yang5 crate (libyang 5.4.9) and refreshing example YANG modules/libraries.
2. feat(yang-push): improve OTEL metrics for validation actor — overhauling the validation actor's OpenTelemetry metrics.
3. feat(collector): migrate kafka-yang-consumer example to yang-rs-5 — migrating the kafka-yang-consumer example from yang4 to yang5.

⚠️ Important: yang5 introduces ANYDATA_STRICT validation (LYD_PARSE_ANYDATA_STRICT flag in libyang) which enforces validation inside anydata subtrees. Existing deployments may see previously-accepted push-update messages now rejected and dropped. Monitor the netgauze.yang_push.validation.dropped{reason="validation_failed"} counter after upgrading.

⚠️ Important: libyang ANYDATA_STRICT currently enables schema-aware parsing and type validation inside anydata subtrees (e.g. invalid leaf values/types are rejected), but it does not yet perform full semantic YANG validation there. Some constraints, such as missing mandatory leaves, are still not enforced inside anydata content. See CESNET/libyang#2548

Changes

yang4 → yang5 migration

• Replaced the yang4 dependency with yang5 (libyang 5.4.9) in the validation actor and the kafka-yang-consumer example in the collector crate - adjusted all code to the updated API and enforcing ANYDATA_STRICT validation
• Revisited and updated example yang modules and yang-lib.xml files in assets. They were incomplete and relying on deprecated yang-modules and now with full anydata_strict validation tests were failing
• Include additional tests covering more validation scenarios
• Address a minor corner case: added schema_fetch_pending tracking on CachedSubscription to detect duplicate SubscriptionStarted/SubscriptionModified packets arriving while a schema lookup is still in-flight, buffering them instead of forwarding unvalidated

OTEL metrics overhaul

• Consolidated and renamed existing metrics
• Replaced several separate drop counters with a single dropped counter tagged by a reason attribute (decode_error, buffer_full_subscription, buffer_full_peer, validation_failed, incomplete_subscription_started, no_subscription_id, send_error).
• Replaced separate cache-request counters with a single cache_lookups counter tagged by a by attribute (subscription_info, subscription_id).
• Added a reason tag to the skipped counter, distinguishing why validation was skipped: no_library, context_failed, and no_subscription_info.
• Add flag for decode_message to avoid double counting of decoded messages
• Add pending gauge tracking packets pending in the queue awaiting reprocessing
• Add gauges tracking how many peers and subscriptions are present in the actor cache

[Copilot]

Pull request overview

This PR upgrades the YANG-Push validation and example consumer code to yang5 (libyang 5.4.9), revamps OpenTelemetry metrics emitted by the validation actor, and refreshes bundled example YANG modules / YANG library XML to work under stricter ANYDATA_STRICT validation.

Changes:

• Migrated yang-push validation actor and kafka-yang-consumer example from yang4 → yang5, enabling ANYDATA_STRICT where applicable.
• Overhauled validation actor OTEL metrics: renamed/consolidated instruments and added new gauges (pending, cached peers/subscriptions).
• Updated bundled example YANG library XML and modules; added/expanded validation actor tests (including schema-fetch in-flight buffering behavior).

Reviewed changes

Copilot reviewed 17 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
crates/yang-push/src/validation/mod.rs	Main migration to yang5, new strict validation behavior, buffering corner-case handling, and OTEL metrics redesign + tests.
crates/yang-push/src/lib.rs	Removes now-obsolete OTEL constants used by the previous metrics scheme.
crates/yang-push/Cargo.toml	Switches dependency from yang4 to yang5.
crates/collector/examples/kafka-yang-consumer.rs	Migrates example validator to yang5 and enables ANYDATA_STRICT.
crates/collector/Cargo.toml	Switches dependency from yang4 to yang5.
Cargo.toml	Replaces workspace yang4 dependency with yang5.
Cargo.lock	Updates lockfile for yang5 / libyang5-sys and related dependency graph changes.
assets/yang/ietf-telemetry-message/yang-lib-full.xml	Refreshes example YANG library to include required modules/structure for stricter validation.
assets/yang/ietf-telemetry-message/modules/ietf-yp-observation@2026-05-11.yang	Adds observation-time augmentation module used by updated examples.
assets/yang/ietf-telemetry-message/modules/ietf-yang-push-revision@2025-12-07.yang	Updates example revision/version augmentation module definitions.
assets/yang/ietf-telemetry-message/modules/ietf-system-capabilities@2022-02-17.yang	Formatting/whitespace adjustments in bundled module copy.
assets/yang/ietf-telemetry-message/modules/ietf-distributed-notif@2026-04-13.yang	Adds distributed notification augmentation module used by updated examples.
assets/yang/ietf-interfaces/yang-lib.xml	Refreshes example YANG library for interface payload validation under strict parsing.
assets/yang/ietf-interfaces/modules/ietf-yp-observation@2026-05-11.yang	Adds observation-time augmentation module for interfaces example set.
assets/yang/ietf-interfaces/modules/ietf-yang-push-revision@2025-12-07.yang	Updates example revision/version augmentation module definitions for interfaces set.
assets/yang/ietf-interfaces/modules/ietf-system-capabilities@2022-02-17.yang	Formatting/whitespace adjustments in bundled module copy.
assets/yang/ietf-interfaces/modules/ietf-interfaces@2018-02-20.yang	Updates bundled ietf-interfaces module copy (NMDA-era model + legacy deprecations).
assets/yang/ietf-interfaces/modules/ietf-distributed-notif@2026-04-13.yang	Adds distributed notification augmentation module for interfaces example set.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 18 changed files in this pull request and generated 2 comments.

Suppressed comments (10)

assets/yang/ietf-telemetry-message/yang-lib-full.xml:52

• This <namespace> value is split across lines, which introduces leading whitespace/newlines into the element text. That makes the namespace URI technically different (and potentially invalid), and may break YANG library parsing / module resolution.

      <namespace>
        urn:ietf:params:xml:ns:yang:ietf-notification-capabilities</namespace>

assets/yang/ietf-telemetry-message/yang-lib-full.xml:142

• This <namespace> value is split across lines, which introduces leading whitespace/newlines into the element text. That makes the namespace URI technically different (and potentially invalid), and may break YANG library parsing / module resolution.

      <namespace>
        urn:ietf:params:xml:ns:yang:ietf-subscribed-notif-receivers</namespace>

assets/yang/ietf-interfaces/yang-lib.xml:60

• This <namespace> value is split across lines, which introduces leading whitespace/newlines into the element text. That makes the namespace URI technically different (and potentially invalid), and may break YANG library parsing / module resolution.

      <namespace>
        urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications</namespace>

assets/yang/ietf-interfaces/yang-lib.xml:117

• This <namespace> value is split across lines, which introduces leading whitespace/newlines into the element text. That makes the namespace URI technically different (and potentially invalid), and may break YANG library parsing / module resolution.

      <namespace>
        urn:ietf:params:xml:ns:yang:ietf-subscribed-notif-receivers</namespace>

assets/yang/ietf-telemetry-message/modules/ietf-yang-push-revision@2025-12-07.yang:154

• This line ends with a tab character. Trailing tabs/whitespace can cause noisy diffs and may trip formatting/lint tooling for YANG sources; please remove it.
  assets/yang/ietf-interfaces/modules/ietf-yang-push-revision@2025-12-07.yang:154
• This line ends with a tab character. Trailing tabs/whitespace can cause noisy diffs and may trip formatting/lint tooling for YANG sources; please remove it.
  assets/yang/ietf-telemetry-message/yang-lib-full.xml:85
• This <namespace> value is split across lines, which introduces leading whitespace/newlines into the element text. That makes the namespace URI technically different (and potentially invalid), and may break YANG library parsing / module resolution.

      <namespace>
        urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications</namespace>

crates/yang-push/src/validation/mod.rs:186

• The doc comment for schema_fetch_pending says it is set only for LookupBySubscriptionInfo, but the implementation also sets it for LookupBySubscriptionId lookups. This makes the state-machine explanation inaccurate for readers and future maintainers.

/// `schema_fetch_pending` is set to `true` when a `LookupBySubscriptionInfo`
/// request is sent and cleared to `false` when the cache actor responds,
/// regardless of whether a schema was found. While it is `true`, duplicate
/// SubscriptionStarted packets are buffered rather than forwarded unvalidated.

assets/yang/ietf-telemetry-message/modules/ietf-yp-observation@2026-05-11.yang:33

• The NETCONF WG URL in the contact block is missing a slash (http:/tools...). This is a typo and results in an invalid URL.

    "WG Web:   <http:/tools.ietf.org/wg/netconf/>

assets/yang/ietf-interfaces/modules/ietf-yp-observation@2026-05-11.yang:33

• The NETCONF WG URL in the contact block is missing a slash (http:/tools...). This is a typo and results in an invalid URL.

    "WG Web:   <http:/tools.ietf.org/wg/netconf/>

[Copilot]

Pull request overview

Copilot reviewed 17 out of 19 changed files in this pull request and generated no new comments.

Suppressed comments (3)

crates/yang-push/src/validation/mod.rs:99

• Module docs say unvalidated packets have content_id = None, but SubscriptionInfo::content_id() is always a ContentId (uses "EMPTY" for placeholders). The optional value that indicates validation is cached_content_id in the actor output tuple, so the docs should reference that to avoid misleading downstream consumers.

//! - **Schema unavailable**: Forwards unvalidated
//!   - `content_id` is `None`; downstream can detect and handle unvalidated
//!     packets

assets/yang/ietf-telemetry-message/modules/ietf-yp-observation@2026-05-11.yang:33

• The contact URL is malformed (http:/... missing a slash), which breaks the WG link in generated docs and makes the module text inaccurate.

    "WG Web:   <http:/tools.ietf.org/wg/netconf/>

assets/yang/ietf-interfaces/modules/ietf-yp-observation@2026-05-11.yang:33

• The contact URL is malformed (http:/... missing a slash), which breaks the WG link in generated docs and makes the module text inaccurate.

    "WG Web:   <http:/tools.ietf.org/wg/netconf/>

[ustorbeck]

The changes proposed in the comments below also have been pushed to:
https://github.com/ustorbeck/NetCalyx/commits/yang5-review
Otherwise it looks good to me.

[rodonile]

@ustorbeck good suggestion with the enum, I cherry picked your commit. I amended it because I did not like to introduce per-message string allocations. Found out about strum_macros::IntoStaticStr, which generates into_str() implementation returning &str for the enum variants thus avoiding the allocations.

In general I like this approach of providing a different branch with some suggested changes, that then we can decide whether to directly cherry-pick and/or adjust.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 19 changed files in this pull request and generated no new comments.

Suppressed comments (1)

crates/yang-push/src/validation/mod.rs:958

• Legacy payload validation via DataTree::parse_op_string does not enable ANYDATA_STRICT, so anydata subtrees may not be strictly validated in the legacy path even though the non-legacy path enforces it. This makes validation behavior inconsistent across message formats and can allow invalid data to slip through when is_legacy is true.

            let validation_result = yang5::data::DataTree::parse_op_string(
                yang_ctx,
                packet.payload(),
                DataFormat::JSON,
                DataParserFlags::STRICT,
                DataOperation::NotificationYang,

[ustorbeck]

then instead of using strum_macros::Display maybe consider to use the same static string table also for the Display trait of SkipReason (used in the trace call at crates/yang-push/src/validation/mod.rs:833)

impl std::fmt::Display for SkipReason {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str((*self).into())
}
}

[ustorbeck]

only one issue with a cache race condition left which already is addressed by PR #35,
approving