Skip to content

Kubernetes locker cannot safely maintain target ownership #982

Description

@pood1e

Bug

The Kubernetes locker rejects valid application target identities and does not bound renewal requests by the remaining Lease validity window. Delayed coordination API requests can leave collection active after ownership has expired.

Type: functionality / correctness / scalability. Impact: a target may fail to acquire ownership or may continue collecting after its Lease expires. This issue consolidates #981.

Reproduction: target identity

Run gNMIc with the k8s locker, its API server, an ordinary subscription named probe, and the documented Lease permissions. Submit this generic example:

curl -X POST http://localhost:7890/api/v1/config/targets \
  -H 'Content-Type: application/json' \
  -d '{"name":"Router-A","address":"192.0.2.1:57400","insecure":true,"subscriptions":["probe"]}'
curl -X POST http://localhost:7890/api/v1/targets/Router-A

The target API accepts the identity, but Lease creation fails because the derived Kubernetes resource name contains uppercase characters. The example address is reserved for documentation; the failure occurs before dialing. Use the runtime API for this case because Viper normalizes YAML map keys.

The encoding only replaces / with - and also uses the result as a label key. Other legal application identities can exceed Kubernetes name constraints or collide after replacement.

Renewal failure mechanism

KeepLock makes synchronous API requests without a request deadline tied to Lease validity. Requests delayed by the client limiter or API server can block renewal and delay the ownership-loss notification. Lock also captures its acquisition timestamp before those requests complete. Query and renewal traffic share the same client request budget.

These paths are in Lock, KeepLock and Unlock and IsLocked/List.

Expected behavior

  • Preserve application identities with bounded, collision-resistant Lease names and original-key metadata.
  • Bound renewal requests and promptly cancel collection when ownership cannot be maintained.
  • Release only the owned Lease, with Kubernetes concurrency preconditions.
  • Use a deliberate API request budget and cached ownership queries.
  • Cover identity boundaries, delayed/failed API requests, cancellation and ownership loss with regression tests.

#983 implements these changes. EndpointSlice discovery is handled separately by #980 / #962.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions