Skip to content

Fix Kubernetes locker discovery with EndpointSlices - #980

Open
pood1e wants to merge 2 commits into
openconfig:mainfrom
pood1e:fix/k8s-endpointslice-962
Open

Fix Kubernetes locker discovery with EndpointSlices#980
pood1e wants to merge 2 commits into
openconfig:mainfrom
pood1e:fix/k8s-endpointslice-962

Conversation

@pood1e

@pood1e pood1e commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Kubernetes locker discovery uses the deprecated core/v1 Endpoints API. This changes direct lookup and peer watching to discovery.k8s.io/v1 EndpointSlices selected by Service name.

The client-go informer maintains the slice set across watch reconnects and expired resource versions. Discovery publishes deterministic, deduplicated snapshots, including empty results after deletion; it filters endpoint conditions and formats IPv6 addresses correctly. The guide documents the Service contract and namespaced RBAC. Lease acquisition, renewal and election logic are unchanged.

Local validation of implementation 92a5d658fa2cdb4b418e5e2ab0bdbaf7e67bdd0b:

  • ./tests/run_tests.sh
  • go test -race ./pkg/lockers/k8s_locker ./pkg/app
  • go vet ./pkg/lockers/k8s_locker
  • Pod-replacement and peer-recovery checks with EndpointSlice permissions and no core/v1 Endpoints permission.

Regression tests cover slice aggregation, Service/namespace selection, endpoint conditions, deduplication, add/update/delete, empty discovery, watch closure, resource-version expiry and cancellation. The documentation-only follow-up at 9ea52f6610a5083c3dbd485fd80b0c848f3c90c6 has parsed YAML examples. GitHub Actions requiring maintainer approval remain pending.

Fixes #962.

@pood1e

pood1e commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

The tested discovery implementation is at 92a5d658fa2cdb4b418e5e2ab0bdbaf7e67bdd0b. Head 9ea52f6610a5083c3dbd485fd80b0c848f3c90c6 adds documentation only; its YAML examples were parsed successfully.

Local discovery tests, race tests and Pod-replacement checks passed. Lease lifecycle changes are reviewed separately in #983.

@pood1e

pood1e commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

Additional peer-discovery checks passed with #983: Pod replacement and recovery worked using EndpointSlice permissions without core/v1 Endpoints permissions. This combined-change check supplements the discovery regressions in this PR.

@pood1e

pood1e commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

Final combined deployment validation is now complete for this EndpointSlice change together with #983:

  • Kubernetes v1.36.2+k3s1, 3 nodes, 6 gNMIc replicas distributed 2/2/2.
  • 641 targets and 4,487 subscriptions remained healthy through a 7,204-second steady-state run.
  • Active target ownership remained unique; Remote Write failures, dropped messages, subscription failures, and container restarts had zero increase.
  • Member and leader replacement recovery had already passed in the dedicated failover run (11.458s and 25.639s business-sample recovery, both below the existing 60s limit).
  • The deployed ServiceAccount can read EndpointSlices but has no legacy Endpoints permission.

The PR is mergeable and all GitHub checks are green. Requesting maintainer review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Request: Migrate Kubernetes locker peer discovery from Endpoints to EndpointSlice

1 participant