Skip to content

Add version and upstream license links to THIRD_PARTY_NOTICES.md - #1

Draft
abrarshivani wants to merge 3 commits into
mainfrom
tpn-version-location
Draft

abrarshivani wants to merge 3 commits into
mainfrom
tpn-version-location

Conversation

@abrarshivani

@abrarshivani abrarshivani commented Aug 27, 2026

Copy link
Copy Markdown
Owner

Adds Version and Location columns to THIRD_PARTY_NOTICES.md, replacing the Dependency
column. Location links to the license file in the dependency's own upstream repository, pinned
to the version we redistribute:

Package Version License Location
github.com/NVIDIA/go-nvml/pkg v0.13.3-1 Apache-2.0 LICENSE

This is the same change as NVIDIA/gpu-operator, applied here.

Version was dropped in b911681 ("Remove Go module versions from rendered TPN document") on the
grounds that the notices identify dependencies and their licenses, not an exact build. That is
reversed here, and I want to flag it rather than bury it. Two reasons: a notices file that does
not say which version it describes cannot be matched to a release, and a link into upstream needs
a ref to point at. The churn is real but small, one index row and two bullets per bump.

Every URL in the file was verified by fetching it and comparing its sha256 against the copy under
vendor/. A URL that does not match is never written, so there are no dead links and none point
at the wrong license. 47 URLs across 41 modules.

That includes the secondary license files a module ships alongside its main one, which are easy
to lose: the three golang.org/x/* and google.golang.org/protobuf PATENTS files, and the
LICENSE.libyaml (MIT, for the embedded libyaml port) that sigs.k8s.io/yaml/goyaml.v2 carries
on top of its Apache-2.0 LICENSE. Five license files that were previously dropped are now
reproduced.

What to review

Hand-written, 965 lines:

File Lines
hack/verify-license-urls.sh 226
hack/generate-third-party-notices_test.sh 194
hack/resolve-module-repos.sh 173
hack/license-url-lib.sh 156
hack/license-url-lib_test.sh 96
.github/workflows/third-party-notices-links.yaml 60
hack/test-helpers.sh 45
hack/generate-third-party-notices.sh +175/-47
Makefile +17/-2
hack/license-overrides.tsv 15
.github/workflows/third-party-notices-check.yaml +7 (comment only)

Generated, do not read: THIRD_PARTY_NOTICES.md, hack/license-urls.tsv (52),
hack/module-repos.tsv (45).

hack/verify-license-urls.sh is the one to read. Everything else supports it.

How it works

vendor/ gives the module, the version and the license file names for free, but it does not
record the upstream repository. cyphar.com/go-pathrs actually lives at
github.com/cyphar/go-pathrs, sigs.k8s.io/yaml at github.com/kubernetes-sigs/yaml. Scraping
that out of vendored sources is wrong more often than right, so two committed maps carry it
instead, both machine-generated:

hack/module-repos.tsv maps module to repository. Resolution is the Go module proxy's Origin,
then the go-import meta tag that go get itself uses, then the github.com/<org>/<repo> path
shape. Nothing is hand-written. It is keyed by module and not by version, so a bump does not
invalidate it.

hack/license-urls.tsv maps module, version and license path to a verified URL. A row is written
only when the bytes at that URL hash identically to the vendored copy. Probing for a 200 is not
enough: it cannot tell a correct link from one that returns 200 for the wrong license.

Both are produced out of band by make third-party-notices-repos and make third-party-notices-urls, which need network. make third-party-notices reads them offline, so
make check-third-party-notices stays hermetic and cannot flap on a proxy that withholds
Origin.

Scope is unchanged: the verifier reuses the generator's own collection, so it covers the packages
go-licenses attributes to ./cmd/..., and CGO_ENABLED=1 still applies for the reason the
generator already documents. Modules vendored only for tests or build tooling are not
redistributed and are not listed.

License files are now enumerated from vendor/ rather than from the go-licenses save output,
because that output keeps only the one file it classifies as the license per package and drops
the rest.

hack/license-overrides.tsv corrects the License column where go-licenses under-reports it. Two
packages here ship a license document holding more than one license:
sigs.k8s.io/yaml/goyaml.v2 carries LICENSE (Apache-2.0) alongside LICENSE.libyaml (MIT),
and gopkg.in/yaml.v3's single LICENSE has a full-text MIT section plus a short-form Apache-2.0
grant. go-licenses classifies each as a single license, so both read Apache-2.0 / MIT from the
override instead. It is curated by hand rather than detected, because scanning license text
cannot tell BSD-2-Clause from BSD-3-Clause and a wrong addition to this file is worse than an
omission. Generation fails if an override names a package that is no longer in the index, so the
file cannot rot unnoticed.

.github/workflows/third-party-notices-links.yaml re-verifies every URL weekly. Links are proven
correct when written, but upstream can retag or archive a repository afterwards and no offline
gate can see that.

Note for dependency bumps

A version change now needs two commands, because a verified URL contains the version:

make third-party-notices-urls   # network
make third-party-notices        # offline

Dependabot cannot do the first on its own. Its bump job needs wiring, or a human runs it. This is
the direct cost of requiring every link to be verified rather than derived.

Testing

make test-tools runs the new bash suites, 54 assertions across two files, and is now part of
CHECK_TARGETS so it runs in CI.

Verified by hand:

  • 33 index rows, four columns, no floating HEAD or branch refs
  • all 33 package versions match vendor/modules.txt
  • regeneration is byte-deterministic across runs
  • the gate fails closed two ways: removing the go-nvml rows from hack/license-urls.tsv makes
    make third-party-notices exit non-zero naming that module, and a license-overrides.tsv row
    for a package not in the index does the same

THIRD_PARTY_NOTICES.md now carries a Version and a Location column instead of
the Dependency column. Location links to the license file in the dependency's
own upstream repository, pinned to the version we redistribute:

| Package | Version | License | Location |
|---------|---------|---------|----------|
| `github.com/NVIDIA/go-nvml/pkg` | v0.13.3-1 | Apache-2.0 | [LICENSE](https://github.com/NVIDIA/go-nvml/blob/v0.13.3-1/LICENSE) |

Version was dropped in b911681 on the grounds that the notices identify
dependencies and their licenses, not an exact build. That is reversed here: a
notices file that does not say which version it describes cannot be matched to
a release, and a link into upstream needs a ref to point at. The churn per bump
is one index row and two bullets.

Every URL was verified by fetching it and comparing its sha256 against the copy
under vendor/. A URL that does not match is never written, so no link is dead
and none points at the wrong license. 47 URLs across 41 modules.

vendor/ gives the module, the version and the license file names for free, but
not the upstream repository: cyphar.com/go-pathrs lives at
github.com/cyphar/go-pathrs, sigs.k8s.io/yaml at github.com/kubernetes-sigs/yaml.
Two committed maps carry that instead, both machine-generated.

hack/module-repos.tsv maps module to repository, resolved from the Go module
proxy's Origin, then the go-import meta tag that go get itself uses, then the
github.com/<org>/<repo> path shape. It is keyed by module and not by version,
so a bump does not invalidate it.

hack/license-urls.tsv maps module, version and license path to a verified URL.
A row is written only when the bytes at that URL hash identically to the
vendored copy. Probing for a 200 is not enough: it cannot tell a correct link
from one that returns 200 for the wrong license.

Both are produced out of band by 'make third-party-notices-repos' and 'make
third-party-notices-urls', which need network. 'make third-party-notices' reads
them offline, so 'make check-third-party-notices' stays hermetic.

License files are now enumerated from vendor/ rather than from the go-licenses
save output, because that output keeps only the one file it classifies as the
license per package and drops the rest. That recovers five files, including the
three golang.org/x PATENTS files and the LICENSE.libyaml that
sigs.k8s.io/yaml/goyaml.v2 ships alongside its Apache-2.0 LICENSE.

hack/license-overrides.tsv corrects the License column where go-licenses
under-reports it. Two packages ship a license document holding more than one
license, so they read Apache-2.0 / MIT from the override. It is curated by hand
rather than detected, because scanning license text cannot tell BSD-2-Clause
from BSD-3-Clause and a wrong addition is worse than an omission. Generation
fails if an override names a package no longer in the index, so it cannot rot
unnoticed.

third-party-notices-links.yaml re-verifies every URL weekly. Links are proven
correct when written, but upstream can retag or archive a repository afterwards
and no offline gate can see that.

A version change now needs two commands, because a verified URL contains the
version:

    make third-party-notices-urls   # network
    make third-party-notices        # offline

Dependabot cannot do the first on its own; its bump job needs wiring, or a
human runs it. That is the direct cost of requiring every link to be verified
rather than derived.

'make test-tools' runs the new bash suites, 67 assertions across two files, and
is part of CHECK_TARGETS so it runs in CI.

Transient failures fetching a license blob are retried. go.googlesource.com
returns 503 and 429 under the per-file loop this drives, and the fail-closed
gate would otherwise treat a rate-limited response as link rot; the weekly link
check drives the same loop. A 404 still fails on the first request, so a real
miss costs one request per candidate. fetch_retry moves into license-url-lib.sh
as http_fetch_to_file so both resolvers share one retry and status policy, and
it writes to a file because command substitution strips the trailing newline
that a license file's sha256 depends on.

Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
…ships

The application stage copies the extracted deb and rpm payloads into the image,
so it ships libnvidia-container.so, libnvidia-container.a,
libnvidia-container-go.so and nvidia-container-cli alongside this repository's
own commands. Those objects carry third-party code that the notices did not
mention: five Go modules linked into libnvidia-container-go.so, and three C
libraries statically linked into libnvidia-container.so.

The Go modules join the existing index rather than forming a second table. What
ships is one filesystem, so a module both trees pull at a different version is
two honest rows: runtime-spec appears at v1.2.0 and v1.3.0, and x/sys at the
two package roots go-licenses attributes for each tree. The tree a row was
read from moves into the row as a sixth field, since it is now a property of
the row rather than of the table.

The C libraries get their own section. elftoolchain and nvidia-modprobe ship no
license file of their own, so their terms are the per-file copyright blocks
scraped from the sources actually compiled in. libtirpc does ship COPYING, and
its Location is checked by fetching it and comparing it byte for byte with the
copy in the archive. nvidia-modprobe's COPYING is deliberately not linked: it
is GPL-2.0 and covers the nvidia-modprobe binaries, which are not built or
shipped here, while the modprobe-utils sources that are compiled in are MIT.

The dependency set is derived from the submodule at its pinned commit, so a
bump shows up as a diff here rather than changing silently.

Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
Absorbing libnvidia-container's C dependencies brought over the Location
column but not the Source column beside it, so the table stopped naming the
archive the build downloads and links in. The index already carried the URL and
the per-dependency sections already printed it; only the table had lost it.

That column carries more weight here than a duplicate link would suggest. Two
of the three dependencies have no license file to point at: elftoolchain 0.7.1
ships none, and nvidia-modprobe's COPYING is the GPL-2.0 covering binaries this
repository does not build. For those two the Location cell is a sentence rather
than a link, which left the table with no pointer at all to code that is
statically linked into the shipped libraries.

The section prose now defines both columns rather than only Location.

Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant