feat(release): score dependency floors and cargo features in semver-level

[iunanua]

What does this PR do?

semver-level.sh combined cargo-semver-checks and cargo-public-api. Both read rustdoc and neither reads a manifest, so two semver-relevant manifest changes came out as patch:

• a raised dependency requirement floor (http = "1" → "1.1"): every rustdoc signature is byte-identical, yet a consumer pinned to the old version can no longer resolve the crate — conventionally a minor;
• an added cargo feature: cargo-semver-checks has no lint for an addition at all, and a feature adds no rustdoc item unless it happens to gate one.

Two passes are added, both reading the manifest through cargo metadata:

pass	reports
2b) dependency requirement floors	minor when the lowest version a requirement admits goes up
2c) cargo feature surface	minor on an added feature; major on one removed, or dropped from the set default reaches

The second commit is a consequence of the first: build moves into the permissive PR-title type list, because scoring a dependency floor as minor would otherwise make build(deps): bump foo from 1.0 to 1.1 — the conventional title for a dependency change — fail the type rule.

The third commit is what lets CI reach any of it. detect-changes selected a crate only when a file under its own directory changed, so a PR editing just the root manifest's [workspace.dependencies] selected nothing: has_rust_changes came out false, both the semver-check and validate jobs were skipped, and every PR title was accepted — a docs: title included. Since this workspace declares dependencies version-only at the root and members inherit them with workspace = true, that is the ordinary shape of a floor raise here, which left the pass above unreachable from CI. semver-level.sh grows a --list-affected mode naming every member whose dependency requirements, dependency feature selections or own feature surface moved between two revisions, and detect-changes adds those crates to the ones it finds by path.

The last three commits answer review findings against the manifest readers. None of them changes what any pass scores on this workspace today:

• Default membership is the closure, not a direct listing. 2c asked whether default lists a feature. So a behaviour-preserving refactor — default = ["foo"] becoming default = ["bundle"] with bundle = ["foo"] — read as foo losing its default and scored major, while the inverse, emptying a bundle that default still lists, is a real break that read as no change at all. Rows now carry the closure over the feature graph, which is what a consumer writing default-features = true actually gets.
• Dependency rows now carry the feature selection. They recorded only the requirement, so a root entry that merely gained a feature, or stopped disabling default features, left every inheriting member's facts identical: --list-affected named nobody and, with only the root manifest touched, the jobs were skipped again — the same shape of gap the third commit fixes for floors.
• Exclusive bounds are normalized, not tagged. req_min appended an exclusivity bit to the version a bound states rather than advancing past it, so > was read as admitting the version it excludes. >1.2 admits nothing below 1.3.0, so >=1.2.1 → >1.2 read as a lowering rather than the raise it is; and >1.2.3 → the equivalent >=1.2.4 read as a raise. Bounds now advance to the first version they admit, which drops the flag and returns version_gt to a plain triple.

Motivation

Reviewing release proposal #2482 surfaced libdd-capabilities proposed as patch, when #2350 had moved its http requirement from ^1 to the workspace entry at ^1.1. That is a minor, and nothing in the pipeline could see it: the version had moved into [workspace.dependencies], so even a textual diff of the crate's own Cargo.toml shows only { workspace = true } with no version in it.

Additional Notes

Points reviewers may want to weigh:

• Read through cargo metadata, not Cargo.toml. That is what resolves { workspace = true } to the version the root manifest declares, and what surfaces the implicit feature an optional = true dependency creates.
• Dependencies are keyed by .rename as well as name/kind/target. A crate aliasing two versions of one package emits an identical name, kind and target for both; keyed without the alias, the second matches the first's baseline requirement and an unchanged pair reads as a raise.
• An exclusive bound is advanced to the first version it admits, > excluding everything up to and including the components it states: >1.2.3 starts at 1.2.4, >1.2 at 1.3.0 and >1 at 2.0.0. So >=1.2.3 → >1.2.3 reads as the narrowing it is, >=1.2.1 → >1.2 as the raise it is, and >1.2.3 → the equivalent >=1.2.4 as no change at all. A pre-release bound is the exception and stays where it is: >1.2.3-rc.1 admits 1.2.3 itself, so the release version this reader already compares is that bound's exact floor, and advancing it would overstate the requirement. req_min was checked against semver 1.0.28's own VersionReq::matches over 33 requirement forms — every operator at one, two and three components, the four shapes this workspace uses, and both pre-release cases — and agrees on all of them.
• A raised major floor is capped at minor on purpose. Whether that forces the dependent to major is release-version-major-bumps.sh's decision, made with the dependency graph this script cannot see; max_level() lets it win from there.
• The two passes stop at different points. 2b can only report minor, so minor ends it. 2c can report major, so it runs on until major — which matters for a crate with no library target, where cargo-semver-checks is skipped entirely and nothing else watches the feature table.
• Feature removals are a backstop, not the primary check. For a library crate feature_missing and feature_not_enabled_by_default get there first; both were verified to fail the run rather than merely warn, using a two-crate repro against the pinned invocation. The closure above is what keeps the backstop from contradicting them: cargo-semver-checks resolves the default set transitively, so keyed on direct membership this pass would have reported major over a refactor the lint passes.
• Dependency feature facts are selected on, not scored. Enabling a feature in a dependency can change this crate's own API — an item reached through a glob re-export appears — but the manifest cannot say whether it did, and the passes that read rustdoc can. So the columns widen what --list-affected names and assign no level of their own; 2b's key stops before them, so a feature move never reads as a requirement change.
• Deliberately not scored: dev-dependencies, a widened requirement, a dependency added or removed outright, and a change to what a feature enables for its own sake (the two *_enables_feature lints' job) — that last reaches 2c only where it moves the set default reaches.

On the detection commit:

• --list-affected lives in semver-level.sh rather than a script of its own, because it reuses manifest_facts_at_rev — now able to read the whole workspace in one extraction — so what the detector considers a manifest fact cannot drift from what the passes score. A detector considering fewer facts silently selects nothing and reports "no change", which is the failure this commit fixes; there are no shell tests to catch that drift. 19 lines are new, the 51 they lean on are shared.
• The mode compares from git merge-base, matching the base...HEAD the changed-file search uses. Measured from the baseline tip instead, a branch behind the baseline reports every crate that moved on the baseline since the fork, putting another branch's changes on this one's report — and since scoring uses the tip, a feature added on the baseline would read as a removal, i.e. a spurious major.
• Publishability stays the caller's business. The mode lists workspace members; detect-changes keeps its own publish filter, so a publish = false member such as libdd-agent-client shows up in the list and is dropped where that filter already lives.
• Cost. A crate selected only by moved facts still gets the full treatment, cargo-semver-checks and cargo-public-api both building rustdoc, so a one-line root bump now costs ~12 of those instead of zero, and a root entry that only gains a feature costs one per inheritor too — 20 for serde_json. That is deliberate: a changed floor or a newly enabled dependency feature can alter a crate's API through re-exports. If it bites, the cheap follow-up is a mode running only passes 2b/2c for those crates. Release-proposal PRs are unaffected — they carry skip-pr-title-semver-check, and each member's version is literal, so the path search already finds them.
• The release path reports the same change rather than releasing for it. commits-since-release.sh selects a crate's commits with git log "$COMMIT_RANGE" -- "$CRATE_PATH", so a root-only floor raise leaves every inheriting crate with no commits and release-version-bumps.sh defers it. That is the right outcome, not a second instance of the bug above: the crate's code is unchanged, the requirement its published version states is still true of that code, and requirements are minimums, so a consumer combining it with a freshly published sibling that asks for the raised floor resolves the newer dependency anyway. Deferring moves neither the version nor the tag, so the raise stays in range and the crate's next release is still scored a minor for it — late, not wrong — while the case where a crate's code does need the raised version always arrives with a change to a file of its own, which the path search already finds. What is missing there is only the saying so: a deferred crate is recorded as level none and reads as "nothing happened". A follow-up reports those moves at the deferral point instead of releasing 12 crates that needed nothing.

Selection, each case a root-manifest edit and nothing else:

root edit	before	after
bytes floor 1.11 → 1.12 (12 members inherit it)	0, both jobs skipped	exactly the 12 inheritors
+ a feature added to one crate	that crate, by path	same, plus the 12
serde_json gains a feature (20 members inherit it outside dev-dependencies)	0, both jobs skipped	exactly those 20
serde_json stops disabling default features	0, both jobs skipped	exactly those 20
baseline unchanged	—	none

Scoring, over the same revisions, is unmoved by any of the three follow-ups: 2b's projection is byte-identical across all 698 dependency rows of the workspace, the 217 feature rows are byte-identical too, and all 149 distinct requirement strings in the workspace keep the floors they had — every bound here being inclusive with a full triple, the normalization has nothing to advance. So the floor and feature passes see exactly what they saw before. Both manifest passes also produce byte-identical output after the field shift in the shared reader (bytes (normal): ^1.11 -> ^1.12; Cargo feature added / added: new-thing). Scoring one of the 20 selected crates end to end reports patch — "No public API changes detected" — rather than a level invented from the new columns.

The closure was checked against cargo-semver-checks 0.47.0 and 0.48.0, the pinned version, on a two-crate repro: feature_not_enabled_by_default passes the behaviour-preserving refactor and fails the inverse, exactly as the closure does, and on libdd-common's real feature graph the pass and the lint name the same eight features. The refactor now scores minor for the genuinely new feature name rather than major.

Everything above runs under RUSTUP_TOOLCHAIN=1.92.0 as the workflow does. shellcheck on the script and on the extracted run: block (actionlint's -e SC2086 profile) reports nothing that was not already there.

🤖 Generated with Claude Code

[datadog-datadog-prod-us1-2]

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 78.07% (-0.03%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 054ba6d | Docs | View more details | Give us feedback!}

[dd-octo-sts]

Artifact Size Benchmark Report

aarch64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so	9.02 MB	9.02 MB	0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a	95.83 MB	95.83 MB	0% (0 B) 👌

aarch64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so	12.18 MB	12.18 MB	0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a	107.21 MB	107.21 MB	0% (0 B) 👌

libdatadog-x64-windows

Artifact	Baseline	Commit	Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll	29.00 MB	29.00 MB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib	96.08 KB	96.08 KB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb	191.41 MB	191.35 MB	--.02% (-56.00 KB) 💪
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib	817.16 MB	817.16 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll	9.68 MB	9.68 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib	96.08 KB	96.08 KB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb	27.45 MB	27.45 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib	55.48 MB	55.48 MB	0% (0 B) 👌

libdatadog-x86-windows

Artifact	Baseline	Commit	Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll	25.35 MB	25.35 MB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib	97.58 KB	97.58 KB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb	196.62 MB	196.59 MB	--.01% (-24.00 KB) 💪
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib	803.21 MB	803.21 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll	7.50 MB	7.50 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib	97.58 KB	97.58 KB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb	29.56 MB	29.56 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib	52.44 MB	52.44 MB	0% (0 B) 👌

x86_64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a	85.82 MB	85.82 MB	0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so	10.02 MB	10.02 MB	0% (0 B) 👌

x86_64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a	101.69 MB	101.69 MB	0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so	12.24 MB	12.24 MB	0% (0 B) 👌

[pr-commenter]

Benchmarks

Comparison

Benchmark execution time: 2026-09-21 08:15:57

Comparing candidate commit 054ba6d in PR branch igor/versioning/semver-level-dep-and-feat with baseline commit d3b93c6 in branch main.

📊 Benchmarking dashboard

Found 3 performance improvements and 1 performance regressions! Performance is the same for 162 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

• 🟩 = significantly better candidate vs. baseline
• 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:thread_cpu/profiler_attached/fast_path_system/4096

• 🟥 execution_time [+13.619ns; +13.734ns] or [+14.979%; +15.106%]

scenario:thread_cpu/profiler_attached/slow_path_system/4096

• 🟩 execution_time [-8.132ns; -8.043ns] or [-5.228%; -5.172%]

scenario:trace_buffer/2_senders/no_delay

• 🟩 execution_time [-188.495µs; -169.797µs] or [-10.877%; -9.798%]
• 🟩 throughput [+113951.962op/s; +126960.504op/s] or [+10.964%; +12.216%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:datadog_sample_span/parent_not_sampled_short_circuit/allocated_bytes

• unstable execution_time [-0.000ns; +0.000ns] or [+556.313%; -556.007%]

scenario:datadog_sample_span/parent_sampled_short_circuit/allocated_bytes

• unstable execution_time [-0.000ns; +0.000ns] or [+568.339%; -561.707%]

scenario:glob_matcher/ascii_case_insensitive_match/allocated_bytes

• unstable execution_time [-0.000ns; +0.000ns] or [+555.735%; -555.735%]

scenario:glob_matcher/ascii_exact_match/allocated_bytes

• unstable execution_time [-0.000ns; +0.000ns] or [+556.872%; -556.270%]

scenario:glob_matcher/ascii_exact_miss/allocated_bytes

• unstable execution_time [-0.000ns; +0.000ns] or [+555.735%; -555.735%]

scenario:glob_matcher/ascii_wildcard_backtrack_match/allocated_bytes

• unstable execution_time [-0.000ns; +0.000ns] or [+554.961%; -555.370%]

scenario:glob_matcher/ascii_wildcard_heavy_backtrack/allocated_bytes

• unstable execution_time [-0.000ns; +0.000ns] or [+556.586%; -556.136%]

scenario:glob_matcher/ascii_wildcard_question_match/allocated_bytes

• unstable execution_time [-0.000ns; +0.000ns] or [+559.614%; -557.564%]

scenario:glob_matcher/ascii_wildcard_star_match/allocated_bytes

• unstable execution_time [-0.000ns; +0.000ns] or [+557.278%; -556.462%]

scenario:glob_matcher/star_short_circuit/allocated_bytes

• unstable execution_time [-0.000ns; +0.000ns] or [+567.676%; -561.392%]

Candidate

Omitted due to size.

Baseline

Omitted due to size.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c49fb4f594

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[yannham]

I'm starting to look at at this, but there's something I don't yet understand in the description:

Reviewing release proposal #2482 surfaced libdd-capabilities proposed as patch, when #2350 had moved its http requirement from ^1 to the workspace entry at ^1.1. That is a minor, and nothing in the pipeline could see it: the version had moved into [workspace.dependencies], so even a textual diff of the crate's own Cargo.toml shows only { workspace = true } with no version in it.

Why would raising a dependency from ^1 to ^1.1 be a minor or major change? Even if some dependent pins the version to 1.0 , they will have at worst a duplicated version, but everything should still compile and work fine, right? (unless we re-export some types from this dependency but this is somehow a contrived case).

Also I would expect the semver script to only check for breaking changes. The difference between minor and patch sounds rather subjective (in this case you could fix an urgent security issue by upgrading a dependency 1.0 -> 1.1 and making that a patch release sounds reasonable IMHO). But this one is not only for CI, it is for release? Does it decide if the release should be major or minor automatically, or does it just double check?

[iunanua]

I'm starting to look at at this, but there's something I don't yet understand in the description:

Reviewing release proposal #2482 surfaced libdd-capabilities proposed as patch, when #2350 had moved its http requirement from ^1 to the workspace entry at ^1.1. That is a minor, and nothing in the pipeline could see it: the version had moved into [workspace.dependencies], so even a textual diff of the crate's own Cargo.toml shows only { workspace = true } with no version in it.

Why would raising a dependency from ^1 to ^1.1 be a minor or major change? Even if some dependent pins the version to 1.0 , they will have at worst a duplicated version, but everything should still compile and work fine, right? (unless we re-export some types from this dependency but this is somehow a contrived case).

It is based on an "agreement" regarding the bumps specified here
At the moment, semver-level.sh is not able to detect dependency version or feature changes and this PR is trying to address it. With the added difficulty that dependencies may be defined in the workspace root Cargo.toml

Also I would expect the semver script to only check for breaking changes. The difference between minor and patch sounds rather subjective (in this case you could fix an urgent security issue by upgrading a dependency 1.0 -> 1.1 and making that a patch release sounds reasonable IMHO). But this one is not only for CI, it is for release? Does it decide if the release should be major or minor automatically, or does it just double check?

The script decides the level. It checks for breaking changes but if there are no breaking it tries to resolve if the changes correspond to a minor or to a patch.
As you said, sometimes the level is subjective and in most cases there would be no difference between releasing a minor or a patch from the consumer point of view.
semver-level.sh is used by the CI to check if the PR's title/description is correct (if there is a breaking it must contain a breaking indicator) and also it is used by the release workflow to get the semver level to be applied to a crate.

That said, this PR is kind of optional to increase semver-level.sh coverage in order to detect those changes in the dependencies.
Although I admit that the script tries to solve increasingly complicated cases and that, in the end, it ends up becoming a fiefdom of AI agents.

@yannham

[yannham]

Thanks for the detailed answer!

It is based on an "agreement" regarding the bumps specified here

This section is about adding a new dependency, not bumping an existing one to a new minor version, right? In the end it's not very important as you mention it, but I don't know if I would flag a semver-compatible update as a necessary being at least a minor version bump.

Although I admit that the script tries to solve increasingly complicated cases and that, in the end, it ends up becoming a fiefdom of AI agents

I probably said it elsewhere already, but... rewrite it in Rust 😛 ?

[yannham]

I can't say I'm very confident in my review, given it's a several hundred lines of complicated bash. But I guess the impact is limited (our own ci/release process) and the overall approach is ok

[iunanua]

/merge

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-09-21 08:33:37 UTC ℹ️ Start processing command /merge

---

2026-09-21 08:33:44 UTC ℹ️ MergeQueue: Pull request is not mergeable yet

It will be processed automatically as soon as GitHub reports it as mergeable. View in MergeQueue UI.

• Run /code blockers to see what is blocking it.
• Run /remove to cancel it.

---

2026-09-21 10:13:13 UTC ⚠️ MergeQueue: This merge request was unqueued

igor.unanua@datadoghq.com unqueued this merge request

[iunanua]

/code blockers

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-09-21 10:04:21 UTC ℹ️ Start processing command /code blockers

---

2026-09-21 10:04:22 UTC ℹ️ Devflow:

Checking merge blockers for #2506...

---

2026-09-21 10:04:30 UTC ℹ️ Devflow: /code blockers

No merge blockers detected.

[iunanua]

/merge

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-09-21 14:35:18 UTC ℹ️ Start processing command /merge

---

2026-09-21 14:35:26 UTC ℹ️ MergeQueue: Pull request is not mergeable yet

It will be processed automatically as soon as GitHub reports it as mergeable. View in MergeQueue UI.

• Run /code blockers to see what is blocking it.
• Run /remove to cancel it.

---

2026-09-21 14:36:10 UTC ℹ️ MergeQueue: merge request added to the queue

The expected merge time in main is approximately 48m (p90).

---

2026-09-21 15:20:33 UTC ❌ MergeQueue: The checks failed on this merge request

Tests failed on this commit 9ed82ba:

• Test x86_64-unknown-linux-gnu on centOS7 docker image
• allchecks

What to do next?

• Investigate the failures and when ready, re-add your pull request to the queue!
• If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
• Any question, go check the FAQ.