Add version and upstream license links to THIRD_PARTY_NOTICES.md - #1
Draft
abrarshivani wants to merge 3 commits into
Draft
abrarshivani wants to merge 3 commits into
abrarshivani wants to merge 3 commits into
Conversation
THIRD_PARTY_NOTICES.md now carries a Version and a Location column instead of the Dependency column. Location links to the license file in the dependency's own upstream repository, pinned to the version we redistribute: | Package | Version | License | Location | |---------|---------|---------|----------| | `github.com/NVIDIA/go-nvml/pkg` | v0.13.3-1 | Apache-2.0 | [LICENSE](https://github.com/NVIDIA/go-nvml/blob/v0.13.3-1/LICENSE) | Version was dropped in b911681 on the grounds that the notices identify dependencies and their licenses, not an exact build. That is reversed here: a notices file that does not say which version it describes cannot be matched to a release, and a link into upstream needs a ref to point at. The churn per bump is one index row and two bullets. Every URL was verified by fetching it and comparing its sha256 against the copy under vendor/. A URL that does not match is never written, so no link is dead and none points at the wrong license. 47 URLs across 41 modules. vendor/ gives the module, the version and the license file names for free, but not the upstream repository: cyphar.com/go-pathrs lives at github.com/cyphar/go-pathrs, sigs.k8s.io/yaml at github.com/kubernetes-sigs/yaml. Two committed maps carry that instead, both machine-generated. hack/module-repos.tsv maps module to repository, resolved from the Go module proxy's Origin, then the go-import meta tag that go get itself uses, then the github.com/<org>/<repo> path shape. It is keyed by module and not by version, so a bump does not invalidate it. hack/license-urls.tsv maps module, version and license path to a verified URL. A row is written only when the bytes at that URL hash identically to the vendored copy. Probing for a 200 is not enough: it cannot tell a correct link from one that returns 200 for the wrong license. Both are produced out of band by 'make third-party-notices-repos' and 'make third-party-notices-urls', which need network. 'make third-party-notices' reads them offline, so 'make check-third-party-notices' stays hermetic. License files are now enumerated from vendor/ rather than from the go-licenses save output, because that output keeps only the one file it classifies as the license per package and drops the rest. That recovers five files, including the three golang.org/x PATENTS files and the LICENSE.libyaml that sigs.k8s.io/yaml/goyaml.v2 ships alongside its Apache-2.0 LICENSE. hack/license-overrides.tsv corrects the License column where go-licenses under-reports it. Two packages ship a license document holding more than one license, so they read Apache-2.0 / MIT from the override. It is curated by hand rather than detected, because scanning license text cannot tell BSD-2-Clause from BSD-3-Clause and a wrong addition is worse than an omission. Generation fails if an override names a package no longer in the index, so it cannot rot unnoticed. third-party-notices-links.yaml re-verifies every URL weekly. Links are proven correct when written, but upstream can retag or archive a repository afterwards and no offline gate can see that. A version change now needs two commands, because a verified URL contains the version: make third-party-notices-urls # network make third-party-notices # offline Dependabot cannot do the first on its own; its bump job needs wiring, or a human runs it. That is the direct cost of requiring every link to be verified rather than derived. 'make test-tools' runs the new bash suites, 67 assertions across two files, and is part of CHECK_TARGETS so it runs in CI. Transient failures fetching a license blob are retried. go.googlesource.com returns 503 and 429 under the per-file loop this drives, and the fail-closed gate would otherwise treat a rate-limited response as link rot; the weekly link check drives the same loop. A 404 still fails on the first request, so a real miss costs one request per candidate. fetch_retry moves into license-url-lib.sh as http_fetch_to_file so both resolvers share one retry and status policy, and it writes to a file because command substitution strips the trailing newline that a license file's sha256 depends on. Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
abrarshivani
force-pushed
the
tpn-version-location
branch
from
August 27, 2026 19:37
4ba435a to
5040ffd
Compare
…ships The application stage copies the extracted deb and rpm payloads into the image, so it ships libnvidia-container.so, libnvidia-container.a, libnvidia-container-go.so and nvidia-container-cli alongside this repository's own commands. Those objects carry third-party code that the notices did not mention: five Go modules linked into libnvidia-container-go.so, and three C libraries statically linked into libnvidia-container.so. The Go modules join the existing index rather than forming a second table. What ships is one filesystem, so a module both trees pull at a different version is two honest rows: runtime-spec appears at v1.2.0 and v1.3.0, and x/sys at the two package roots go-licenses attributes for each tree. The tree a row was read from moves into the row as a sixth field, since it is now a property of the row rather than of the table. The C libraries get their own section. elftoolchain and nvidia-modprobe ship no license file of their own, so their terms are the per-file copyright blocks scraped from the sources actually compiled in. libtirpc does ship COPYING, and its Location is checked by fetching it and comparing it byte for byte with the copy in the archive. nvidia-modprobe's COPYING is deliberately not linked: it is GPL-2.0 and covers the nvidia-modprobe binaries, which are not built or shipped here, while the modprobe-utils sources that are compiled in are MIT. The dependency set is derived from the submodule at its pinned commit, so a bump shows up as a diff here rather than changing silently. Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
Absorbing libnvidia-container's C dependencies brought over the Location column but not the Source column beside it, so the table stopped naming the archive the build downloads and links in. The index already carried the URL and the per-dependency sections already printed it; only the table had lost it. That column carries more weight here than a duplicate link would suggest. Two of the three dependencies have no license file to point at: elftoolchain 0.7.1 ships none, and nvidia-modprobe's COPYING is the GPL-2.0 covering binaries this repository does not build. For those two the Location cell is a sentence rather than a link, which left the table with no pointer at all to code that is statically linked into the shipped libraries. The section prose now defines both columns rather than only Location. Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
VersionandLocationcolumns toTHIRD_PARTY_NOTICES.md, replacing theDependencycolumn.
Locationlinks to the license file in the dependency's own upstream repository, pinnedto the version we redistribute:
github.com/NVIDIA/go-nvml/pkgThis is the same change as NVIDIA/gpu-operator, applied here.
Version was dropped in b911681 ("Remove Go module versions from rendered TPN document") on the
grounds that the notices identify dependencies and their licenses, not an exact build. That is
reversed here, and I want to flag it rather than bury it. Two reasons: a notices file that does
not say which version it describes cannot be matched to a release, and a link into upstream needs
a ref to point at. The churn is real but small, one index row and two bullets per bump.
Every URL in the file was verified by fetching it and comparing its sha256 against the copy under
vendor/. A URL that does not match is never written, so there are no dead links and none pointat the wrong license. 47 URLs across 41 modules.
That includes the secondary license files a module ships alongside its main one, which are easy
to lose: the three
golang.org/x/*andgoogle.golang.org/protobufPATENTSfiles, and theLICENSE.libyaml(MIT, for the embedded libyaml port) thatsigs.k8s.io/yaml/goyaml.v2carrieson top of its Apache-2.0
LICENSE. Five license files that were previously dropped are nowreproduced.
What to review
Hand-written, 965 lines:
hack/verify-license-urls.shhack/generate-third-party-notices_test.shhack/resolve-module-repos.shhack/license-url-lib.shhack/license-url-lib_test.sh.github/workflows/third-party-notices-links.yamlhack/test-helpers.shhack/generate-third-party-notices.shMakefilehack/license-overrides.tsv.github/workflows/third-party-notices-check.yamlGenerated, do not read:
THIRD_PARTY_NOTICES.md,hack/license-urls.tsv(52),hack/module-repos.tsv(45).hack/verify-license-urls.shis the one to read. Everything else supports it.How it works
vendor/gives the module, the version and the license file names for free, but it does notrecord the upstream repository.
cyphar.com/go-pathrsactually lives atgithub.com/cyphar/go-pathrs,sigs.k8s.io/yamlatgithub.com/kubernetes-sigs/yaml. Scrapingthat out of vendored sources is wrong more often than right, so two committed maps carry it
instead, both machine-generated:
hack/module-repos.tsvmaps module to repository. Resolution is the Go module proxy'sOrigin,then the
go-importmeta tag thatgo getitself uses, then thegithub.com/<org>/<repo>pathshape. Nothing is hand-written. It is keyed by module and not by version, so a bump does not
invalidate it.
hack/license-urls.tsvmaps module, version and license path to a verified URL. A row is writtenonly when the bytes at that URL hash identically to the vendored copy. Probing for a 200 is not
enough: it cannot tell a correct link from one that returns 200 for the wrong license.
Both are produced out of band by
make third-party-notices-reposandmake third-party-notices-urls, which need network.make third-party-noticesreads them offline, somake check-third-party-noticesstays hermetic and cannot flap on a proxy that withholdsOrigin.Scope is unchanged: the verifier reuses the generator's own collection, so it covers the packages
go-licensesattributes to./cmd/..., andCGO_ENABLED=1still applies for the reason thegenerator already documents. Modules vendored only for tests or build tooling are not
redistributed and are not listed.
License files are now enumerated from
vendor/rather than from thego-licenses saveoutput,because that output keeps only the one file it classifies as the license per package and drops
the rest.
hack/license-overrides.tsvcorrects the License column where go-licenses under-reports it. Twopackages here ship a license document holding more than one license:
sigs.k8s.io/yaml/goyaml.v2carriesLICENSE(Apache-2.0) alongsideLICENSE.libyaml(MIT),and
gopkg.in/yaml.v3's singleLICENSEhas a full-text MIT section plus a short-form Apache-2.0grant. go-licenses classifies each as a single license, so both read
Apache-2.0 / MITfrom theoverride instead. It is curated by hand rather than detected, because scanning license text
cannot tell BSD-2-Clause from BSD-3-Clause and a wrong addition to this file is worse than an
omission. Generation fails if an override names a package that is no longer in the index, so the
file cannot rot unnoticed.
.github/workflows/third-party-notices-links.yamlre-verifies every URL weekly. Links are provencorrect when written, but upstream can retag or archive a repository afterwards and no offline
gate can see that.
Note for dependency bumps
A version change now needs two commands, because a verified URL contains the version:
Dependabot cannot do the first on its own. Its bump job needs wiring, or a human runs it. This is
the direct cost of requiring every link to be verified rather than derived.
Testing
make test-toolsruns the new bash suites, 54 assertions across two files, and is now part ofCHECK_TARGETSso it runs in CI.Verified by hand:
HEADor branch refsvendor/modules.txtgo-nvmlrows fromhack/license-urls.tsvmakesmake third-party-noticesexit non-zero naming that module, and alicense-overrides.tsvrowfor a package not in the index does the same