Skip to content

fix(crossview): roll the app when its OIDC config changes - #3171

Merged
devantler merged 6 commits into
mainfrom
claude/crossview-config-reload
Aug 17, 2026
Merged

fix(crossview): roll the app when its OIDC config changes#3171
devantler merged 6 commits into
mainfrom
claude/crossview-config-reload

Conversation

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Why

Logging in to Crossview fails at Dex with Unregistered redirect_uri. Git is not wrong — Dex
registers the public HTTPS callback and Crossview is configured with the matching value. The running
pod is what disagrees.

Crossview's chart injects every OIDC value as environment variables, and environment is resolved once
when the container starts. Nothing rolled the app when its config changed, so it kept serving the
callback URL it happened to boot with while Dex reconciled the new registration. The two halves of
one change landed a few minutes apart and only one of them took effect.

Measured in production: the config was written at 16:48:08Z, and the container has been running since
16:26:36Z with zero restarts — so it had never read it.

What

Annotate the Crossview Deployment for Reloader, which already runs cluster-wide and is the same
mechanism auth-proxy, oauth2-proxy and homepage use. Config changes now roll the app, so a Dex
registration and the callback the app actually sends cannot drift apart again.

This is the durable half. Unblocking the current session still needs one manual restart, because
the fix only takes effect from the next config change onward.

Part of #2741's rollout — that PR restored the public route and moved the callback back to the public
HTTPS origin; this makes the app actually pick such a change up.

The chart injects every OIDC value as env via env[].valueFrom.configMapKeyRef,
and env is resolved once at container creation, so a crossview-config change
never reaches the running process. Nothing rolled the Deployment on that
change, so the pod kept serving the OIDC_CALLBACK_URL it started with while
Dex reconciled the new registration - and every login failed with an
unregistered redirect_uri.

Measured in prod: crossview-config was written 16:48:08Z, the container has
run since 16:26:36Z with restarts=0, so it had never read the current config.

Annotate the Deployment for Reloader, which is already deployed cluster-wide
and is the same convention auth-proxy, oauth2-proxy and homepage use.
@github-actions

github-actions Bot commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

⚠️MegaLinter analysis: Success with warnings

⚠️ BASH / bash-exec - 4 errors
Results of bash-exec linter (version 5.3.9)
See documentation on https://megalinter.io/9.6.0/descriptors/bash_bash_exec/
-----------------------------------------------

✅ [SUCCESS] .github/scripts/setup-ksail.sh
✅ [SUCCESS] scripts/check-megalinter-version-drift.sh
✅ [SUCCESS] scripts/dr-rebuild-supersession-guard.sh
❌ [ERROR] scripts/ghcr-auth-lib.sh
    Error: File:[scripts/ghcr-auth-lib.sh] is not executable

✅ [SUCCESS] scripts/guard-cilium-homogeneous-device-rollout.sh
✅ [SUCCESS] scripts/guard-kubescape-gate-frameworks.sh
✅ [SUCCESS] scripts/guard-shared-publish-workflow-pin.sh
✅ [SUCCESS] scripts/megalinter-scan-counts.sh
✅ [SUCCESS] scripts/normalize-sarif-paths.sh
❌ [ERROR] scripts/refresh-flux-ghcr-auth-safety.sh
    Error: File:[scripts/refresh-flux-ghcr-auth-safety.sh] is not executable

✅ [SUCCESS] scripts/refresh-flux-ghcr-auth.sh
✅ [SUCCESS] scripts/report-cilium-rollout-gate-suppression.sh
✅ [SUCCESS] scripts/run-ksail-prod-with-pull-auth.sh
✅ [SUCCESS] scripts/summarize-sarif-findings.sh
✅ [SUCCESS] scripts/tests/test-actual-budget-auth-route.sh
✅ [SUCCESS] scripts/tests/test-check-megalinter-version-drift.sh
✅ [SUCCESS] scripts/tests/test-cilium-bandwidth-manager-component.sh
✅ [SUCCESS] scripts/tests/test-cilium-homogeneous-devices-activation.sh
✅ [SUCCESS] scripts/tests/test-cilium-homogeneous-devices-autoscaler-gate.sh
✅ [SUCCESS] scripts/tests/test-cilium-homogeneous-devices-flux-wait.sh
✅ [SUCCESS] scripts/tests/test-cilium-mutual-auth-policy-regressions.sh
✅ [SUCCESS] scripts/tests/test-cilium-mutual-auth-policy.sh
✅ [SUCCESS] scripts/tests/test-cilium-rollout-gate-suppression-signal.sh
✅ [SUCCESS] scripts/tests/test-cnpg-degraded-alert.sh
✅ [SUCCESS] scripts/tests/test-coroot-postgres-scrape-policy.sh
✅ [SUCCESS] scripts/tests/test-crossplane-egress-policy.sh
✅ [SUCCESS] scripts/tests/test-crossplane-sync-exporter.sh
✅ [SUCCESS] scripts/tests/test-dr-rebuild-supersession-guard.sh
✅ [SUCCESS] scripts/tests/test-github-config-role-activation-parity.sh
❌ [ERROR] scripts/tests/test-headlamp-plugin-removal.sh
    Error: File:[scripts/tests/test-headlamp-plugin-removal.sh] is not executable

✅ [SUCCESS] scripts/tests/test-kubescape-gate-frameworks-guard.sh
✅ [SUCCESS] scripts/tests/test-kyverno-admission-vpa.sh
❌ [ERROR] scripts/tests/test-kyverno-umami-mutation-rbac.sh
    Error: File:[scripts/tests/test-kyverno-umami-mutation-rbac.sh] is not executable

✅ [SUCCESS] scripts/tests/test-megalinter-scan-counts-ignorefile.sh
✅ [SUCCESS] scripts/tests/test-normalize-sarif-paths.sh
✅ [SUCCESS] scripts/tests/test-openbao-oidc-role.sh
✅ [SUCCESS] scripts/tests/test-opencost-usage-scraper.sh
✅ [SUCCESS] scripts/tests/test-pvc-prune-safety.sh
✅ [SUCCESS] scripts/tests/test-refresh-flux-ghcr-auth-safety.sh
✅ [SUCCESS] scripts/tests/test-restrict-homepage-service-groups.sh
✅ [SUCCESS] scripts/tests/test-restrict-tenant-secret-stores.sh
✅ [SUCCESS] scripts/tests/test-setup-ksail.sh
✅ [SUCCESS] scripts/tests/test-shared-publish-workflow-pin-guard.sh
✅ [SUCCESS] scripts/tests/test-summarize-sarif-findings.sh
✅ [SUCCESS] scripts/tests/test-umami-provisioning-bootstrap.sh
✅ [SUCCESS] scripts/tests/test-use-prod-stable-api-endpoint.sh
✅ [SUCCESS] scripts/tests/test-validate-image-verifier-liveness.sh
✅ [SUCCESS] scripts/tests/test-verify-published-evidence.sh
✅ [SUCCESS] scripts/update-vendored-operators.sh
✅ [SUCCESS] scripts/use-prod-stable-api-endpoint.sh
✅ [SUCCESS] scripts/validate-alert-coverage.sh
✅ [SUCCESS] scripts/validate-image-verifier-liveness.sh
✅ [SUCCESS] scripts/verify-published-evidence.sh
✅ [SUCCESS] scripts/wait-for-platform-flux-revision.sh
⚠️ REPOSITORY / checkov - 4 errors
2026-08-17 10:20:24,344 [MainThread  ] [ERROR]  YAML error parsing k8s/bases/infrastructure/controllers/kubevirt/kubevirt-operator.yaml: expected a single document in the stream
  in "<unicode string>", line 2, column 1
but found another document
  in "<unicode string>", line 9, column 1
cloudformation scan results:

Passed checks: 0, Failed checks: 0, Skipped checks: 0, Parsing errors: 1

kubernetes scan results:

Passed checks: 2167, Failed checks: 3, Skipped checks: 35

Check: CKV_K8S_40: "Containers should run as a high UID to avoid host conflict"
	FAILED for resource: CronJob.openbao.vault-snapshot
	File: /k8s/bases/infrastructure/vault-backup/cron-job.yaml:23-197
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-37

		Code lines for this resource are too many. Please use IDE of your choice to review the file.
Check: CKV_K8S_40: "Containers should run as a high UID to avoid host conflict"
	FAILED for resource: Job.openbao.vault-snapshot-init
	File: /k8s/bases/infrastructure/vault-backup/job.yaml:23-191
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-37

		Code lines for this resource are too many. Please use IDE of your choice to review the file.
Check: CKV_K8S_38: "Ensure that Service Account Tokens are only mounted where necessary"
	FAILED for resource: CronJob.umami.umami-provision-tenants
	File: /k8s/bases/apps/umami/cron-job.yaml:45-755
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-35

		Code lines for this resource are too many. Please use IDE of your choice to review the file.
secrets scan results:

Passed checks: 0, Failed checks: 1, Skipped checks: 0

Check: CKV_SECRET_6: "Base64 High Entropy String"
	FAILED for resource: HIDDEN_BY_MEGALINTER	File: /k8s/bases/apps/actual-budget/helm-release.yaml:229-230
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/secrets-policies/secrets-policy-index/git-secrets-6

		229 |         tokenExpiration: "ope**********"

github_actions scan results:

Passed checks: 156, Failed checks: 0, Skipped checks: 0
⚠️ SPELL / cspell - 3861 errors
"heredocs",
        "homelab",
        "hostnames",
        "httproute",
        "idempotently",
        "ignorefile",
        "imagetools",
        "imagevalidatingpolicy",
        "imranismail",
        "injective",
        "inspectable",
        "iscsi",
        "italicise",
        "ivpol",
        "jobif",
        "journalled",
        "keylessly",
        "keypair",
        "kprobes",
        "kptr",
        "krew",
        "ksail",
        "ksail's",
        "ksailcd",
        "kubeconfig",
        "kubeconfig's",
        "kubeconform",
        "kubeconform's",
        "kubelet",
        "kubelet's",
        "kubelets",
        "kubelogin",
        "kubescape",
        "kubescape's",
        "kubespan",
        "kubevirt",
        "kubevuln",
        "kustomization",
        "kustomizations",
        "kyverno",
        "letsencrypt",
        "libc",
        "libgnutls",
        "lintable",
        "livez",
        "loadtester",
        "locationless",
        "logfile",
        "lookarounds",
        "lrwxrwxrwx",
        "lserror",
        "lsfail",
        "luks",
        "lycheeignore",
        "machineconfig",
        "materialise",
        "materialised",
        "materialises",
        "maxage",
        "maxbackup",
        "maxsize",
        "maxx",
        "microtime",
        "mikefarah",
        "misconfig",
        "misconfigs",
        "misordered",
        "mistargeted",
        "mitrelike",
        "mktemp",
        "mlock",
        "mutatingpolicies",
        "mutatingwebhookconfigurations",
        "myapp",
        "najsk",
        "neighbour",
        "nenv",
        "netlink",
        "netpol",
        "netpols",
        "neutralises",
        "neutralising",
        "nextjs",
        "nftables",
        "nilnil",
        "nobuckets",
        "nodepod",
        "nodeport",
        "nolabel",
        "nolint",
        "nonexec",
        "nonroot",
        "normalisation",
        "normalise",
        "normalised",
        "normalises",
        "normalising",
        "nosec",
        "nsalike",
        "nullglob",
        "ocirepository",
        "onlycri",
        "onlysystem",
        "openbao",
        "opencost",
        "openfeature",
        "oras",
        "otherplugin",
        "overclaimed",
        "overprovisioning",
        "parallelised",
        "partialgroup",
        "pasteable",
        "permissioning",
        "persistentvolumeclaims",
        "phaseless",
        "pipefail",
        "policyignore",
        "policyreports",
        "portforward",
        "preemptible",
        "preservingly",
        "prioritisable",
        "prioritisation",
        "prioritised",
        "privesc",
        "probeerror",
        "providerconfigs",
        "pseudonymization",
        "pseudonymized",
        "pseudonymizes",
        "publishapp",
        "publishprovider",
        "pushsecret",
        "pushsecrets",
        "pycache",
        "qrbvrml",
        "randomises",
        "rdqwpktr",
        "readyz",
        "reassertions",
        "recognisable",
        "recognisably",
        "recognise",
        "recognised",
        "recolour",
        "reconverges",
        "refreshfluxghcrauth",
        "regenerable",
        "releaserc",
        "rematerialise",
        "rematerialised",
        "replicaset",
        "repoint",
        "repointed",
        "repoints",
        "repositoryrulesets",
        "resizer",
        "restrictor",
        "retabbed",
        "retarget",
        "rmem",
        "rolebindings",
        "rollouts",
        "rshared",
        "rwxr",
        "sanitised",
        "sanitiser",
        "sanitising",
        "sarif",
        "scheckov",
        "schedulability",
        "schedulable",
        "scopeable",
        "scopeless",
        "seccomp",
        "secretbox",
        "secretstore",
        "seedable",
        "serialise",
        "serialised",
        "serverside",
        "serviceaccount",
        "serviceaccounts",
        "sgdisk",
        "shellcheck",
        "shfmt",
        "shopt",
        "shortsha",
        "siderolabs",
        "siderolink",
        "sigstore",
        "skmde",
        "slurpfile",
        "sngle",
        "specnull",
        "spiffe",
        "sprintf",
        "srole",
        "stakater",
        "startswith",
        "statefulset",
        "statemanager",
        "stdlib",
        "stepif",
        "storageclass",
        "strenv",
        "subshell",
        "subtest",
        "summarised",
        "surfaceless",
        "syft",
        "syscall",
        "sysctls",
        "syste",
        "systembare",
        "tagliteral",
        "tagonly",
        "talosconfig",
        "talosctl",
        "tanzu",
        "templatesyncignore",
        "thresholded",
        "tlsv",
        "tmpl",
        "toplevel",
        "tostring",
        "tracepoints",
        "travelled",
        "trixie",
        "trueish",
        "trustd",
        "ture",
        "uids",
        "umami",
        "umami's",
        "unablated",
        "unclickable",
        "uncompilable",
        "unconfigured",
        "uncordon",
        "uncordoned",
        "uncordoning",
        "uncordons",
        "undecoded",
        "undercounts",
        "undispositioned",
        "unenforcing",
        "unevidenced",
        "unexcepted",
        "unfiled",
        "ungated",
        "ungenerated",
        "unifi",
        "uninspected",
        "uninvoked",
        "unioned",
        "unmarshalling",
        "unmarshals",
        "unmodelled",
        "unparseable",
        "unprovisioned",
        "unrecognisable",
        "unrecognised",
        "unrepresentable",
        "unreviewed",
        "unroutable",
        "unrun",
        "unshippable",
        "unskipped",
        "untrackable",
        "unvalidated",
        "unwaited",
        "unwired",
        "upbound",
        "updatekeys",
        "upjet",
        "upstreaming",
        "urlencode",
        "userinfo",
        "userns",
        "ushfn",
        "validatable",
        "validatealertcoverage",
        "validatingwebhookconfigurations",
        "vcunav",
        "velero",
        "virt",
        "volumesnapshot",
        "vpas",
        "vulnerabilitymanifests",
        "vulnerabilitymanifestsummary",
        "vulns",
        "vxlan",
        "wffc",
        "wgpolicyk",
        "wildcarded",
        "wlid",
        "wmem",
        "workloadconfigurationscans",
        "workloadconfigurationscansummary",
        "worktrees",
        "xpkg",
        "yannh",
        "yubikey",
        "yzwvjjmcyfnl",
        "zizmor"
    ]
}


You can also copy-paste megalinter-reports/.cspell.json at the root of your repository

(Truncated to last 6666 characters out of 692763)
⚠️ COPYPASTE / jscpd - 207 errors
ts/test-crossplane-egress-policy.sh [260:25 - 268:19] (9 lines, 52 tokens)
   scripts/tests/test-crossplane-egress-policy.sh [280:24 - 288:19]
Clone found (bash)
 - scripts/tests/test-crossplane-egress-policy.sh [260:25 - 268:18] (9 lines, 51 tokens)
   scripts/tests/test-crossplane-egress-policy.sh [305:31 - 313:18]
Clone found (bash)
 - scripts/tests/test-crossplane-egress-policy.sh [291:13 - 299:8] (9 lines, 52 tokens)
   scripts/tests/test-crossplane-egress-policy.sh [370:18 - 378:14]
Clone found (bash)
 - scripts/tests/test-crossplane-egress-policy.sh [369:22 - 378:15] (10 lines, 63 tokens)
   scripts/tests/test-crossplane-egress-policy.sh [379:33 - 388:15]
Clone found (bash)
 - scripts/tests/test-crossplane-sync-exporter.sh [186:1 - 235:2] (50 lines, 299 tokens)
   scripts/tests/test-kyverno-umami-mutation-rbac.sh [16:1 - 63:2]
Clone found (bash)
 - scripts/tests/test-crossplane-sync-exporter.sh [186:1 - 237:11] (52 lines, 303 tokens)
   scripts/tests/test-opencost-usage-scraper.sh [33:1 - 84:11]
Clone found (bash)
 - scripts/tests/test-crossplane-sync-exporter.sh [221:27 - 238:53] (18 lines, 53 tokens)
   scripts/tests/test-opencost-usage-scraper.sh [68:27 - 85:51]
Clone found (bash)
 - scripts/tests/test-headlamp-plugin-removal.sh [121:89 - 126:22] (6 lines, 65 tokens)
   scripts/tests/test-headlamp-plugin-removal.sh [165:84 - 170:22]
Clone found (bash)
 - scripts/tests/test-refresh-flux-ghcr-auth-safety.sh [348:5 - 356:51] (9 lines, 76 tokens)
   scripts/tests/test-refresh-flux-ghcr-auth-safety.sh [357:5 - 365:51]
Clone found (python)
 - scripts/tests/test_validate_homepage_bookmarks.py [46:57 - 54:54] (9 lines, 58 tokens)
   scripts/tests/test_validate_homepage_bookmarks.py [100:53 - 109:54]
Clone found (go)
 - scripts/validate-dr-signing/main_test.go [1007:13 - 1015:2] (9 lines, 51 tokens)
   scripts/validate-dr-signing/main_test.go [1075:12 - 1083:2]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [564:50 - 569:24] (6 lines, 103 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [1076:31 - 1081:24]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [596:14 - 602:4] (7 lines, 115 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [923:43 - 929:4]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [622:13 - 631:7] (10 lines, 158 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [1205:18 - 1214:7]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [631:1 - 636:8] (6 lines, 93 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [1215:1 - 1220:8]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [708:5 - 713:2] (6 lines, 82 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [894:8 - 899:9]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [709:1 - 719:23] (11 lines, 220 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [986:60 - 997:3]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [713:15 - 724:2] (12 lines, 185 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [814:17 - 825:2]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [760:15 - 771:12] (12 lines, 264 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [969:1 - 982:3]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [823:2 - 829:19] (7 lines, 89 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [1008:7 - 1014:19]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [823:2 - 829:33] (7 lines, 103 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [1390:58 - 1396:33]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [823:2 - 829:4] (7 lines, 74 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [1503:5 - 1509:4]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [916:30 - 921:8] (6 lines, 50 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [943:44 - 948:8]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [1030:22 - 1036:2] (7 lines, 135 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [1589:22 - 1595:2]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [1328:33 - 1336:11] (9 lines, 118 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [1340:130 - 1348:11]
Clone found (go)
 - scripts/validate-eks-ci-role-policy/main_test.go [1535:47 - 1540:2] (6 lines, 166 tokens)
   scripts/validate-eks-ci-role-policy/main_test.go [1615:46 - 1620:2]
Clone found (go)
 - scripts/validate-flux-verify/instance_test.go [109:62 - 126:31] (18 lines, 57 tokens)
   scripts/validate-flux-verify/instance_test.go [161:57 - 178:31]
Clone found (go)
 - scripts/validate-flux-verify/instance_test.go [109:62 - 128:35] (20 lines, 62 tokens)
   scripts/validate-flux-verify/instance_test.go [198:55 - 217:26]
Clone found (python)
 - scripts/validate-naming.py [126:52 - 132:25] (7 lines, 53 tokens)
   scripts/validate-naming.py [171:82 - 177:29]
┌────────┬────────────────┬─────────────┬──────────────┬──────────────┬──────────────────┬───────────────────┐
│ Format │ Files analyzed │ Total lines │ Total tokens │ Clones found │ Duplicated lines │ Duplicated tokens │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ bash   │ 57             │ 18934       │ 74380        │ 37           │ 504 (2.66%)      │ 3452 (4.64%)      │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ go     │ 37             │ 32283       │ 193382       │ 104          │ 858 (2.66%)      │ 7991 (4.13%)      │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ python │ 4              │ 807         │ 5606         │ 2            │ 14 (1.73%)       │ 111 (1.98%)       │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ txt    │ 57             │ 3378        │ 109125       │ 64           │ 2417 (71.55%)    │ 80219 (73.51%)    │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ Total: │ 155            │ 55402       │ 382493       │ 207          │ 3793 (6.85%)     │ 91773 (23.99%)    │
└────────┴────────────────┴─────────────┴──────────────┴──────────────┴──────────────────┴───────────────────┘
Found 207 clones.
HTML report saved to megalinter-reports/copy-paste/jscpd-report.html
ERROR: jscpd found too many duplicates (6.8%) over threshold (0.0%)
time: 1.40s

(Truncated to last 6666 characters out of 43207)
⚠️ MARKDOWN / markdownlint - 65 errors
length Line length [Expected: 400; Actual: 439]
AGENTS.md:105:401 error MD013/line-length Line length [Expected: 400; Actual: 1126]
AGENTS.md:106:401 error MD013/line-length Line length [Expected: 400; Actual: 628]
AGENTS.md:107:401 error MD013/line-length Line length [Expected: 400; Actual: 1774]
AGENTS.md:160:401 error MD013/line-length Line length [Expected: 400; Actual: 649]
AGENTS.md:162:401 error MD013/line-length Line length [Expected: 400; Actual: 971]
AGENTS.md:189:401 error MD013/line-length Line length [Expected: 400; Actual: 970]
AGENTS.md:193:401 error MD013/line-length Line length [Expected: 400; Actual: 660]
AGENTS.md:211:401 error MD013/line-length Line length [Expected: 400; Actual: 1510]
AGENTS.md:279:401 error MD013/line-length Line length [Expected: 400; Actual: 1016]
AGENTS.md:280:401 error MD013/line-length Line length [Expected: 400; Actual: 491]
AGENTS.md:281:401 error MD013/line-length Line length [Expected: 400; Actual: 468]
AGENTS.md:287:401 error MD013/line-length Line length [Expected: 400; Actual: 532]
AGENTS.md:289:401 error MD013/line-length Line length [Expected: 400; Actual: 523]
AGENTS.md:292:401 error MD013/line-length Line length [Expected: 400; Actual: 613]
AGENTS.md:293:401 error MD013/line-length Line length [Expected: 400; Actual: 714]
AGENTS.md:297:401 error MD013/line-length Line length [Expected: 400; Actual: 502]
AGENTS.md:301:401 error MD013/line-length Line length [Expected: 400; Actual: 441]
AGENTS.md:306:401 error MD013/line-length Line length [Expected: 400; Actual: 427]
AGENTS.md:409:401 error MD013/line-length Line length [Expected: 400; Actual: 1139]
AGENTS.md:411:401 error MD013/line-length Line length [Expected: 400; Actual: 1240]
AGENTS.md:427:401 error MD013/line-length Line length [Expected: 400; Actual: 1137]
AGENTS.md:824:401 error MD013/line-length Line length [Expected: 400; Actual: 790]
AGENTS.md:829:401 error MD013/line-length Line length [Expected: 400; Actual: 515]
CLAUDE.md:1 error MD041/first-line-heading/first-line-h1 First line in a file should be a top-level heading [Context: "@AGENTS.md"]
docs/dr/alerting.md:226:28 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/crypto-custody.md:22:389 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/crypto-custody.md:23:264 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/crypto-custody.md:27:35 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/crypto-custody.md:27:161 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/crypto-custody.md:27:239 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/crypto-custody.md:114 error MD024/no-duplicate-heading Multiple headings with the same content [Context: "Custody recommendations"]
docs/dr/crypto-custody.md:245 error MD024/no-duplicate-heading Multiple headings with the same content [Context: "Custody recommendations"]
docs/dr/crypto-custody.md:251 error MD024/no-duplicate-heading Multiple headings with the same content [Context: "What to do if it leaks"]
docs/dr/crypto-custody.md:258 error MD024/no-duplicate-heading Multiple headings with the same content [Context: "What to do if it is *lost* (no..."]
docs/dr/restore-drill.md:42 error MD028/no-blanks-blockquote Blank line inside blockquote
docs/dr/runbook.md:23:102 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/runbook.md:23:487 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/runbook.md:34 error MD028/no-blanks-blockquote Blank line inside blockquote
docs/dr/runbook.md:41 error MD028/no-blanks-blockquote Blank line inside blockquote
docs/dr/runbook.md:50 error MD028/no-blanks-blockquote Blank line inside blockquote
docs/dr/runbook.md:589:92 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/spire-server-ha.md:93 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
docs/dr/velero-cnpg.md:11 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
docs/dr/velero-cnpg.md:56:78 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/velero-cnpg.md:56:166 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/velero-cnpg.md:57:78 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/velero-cnpg.md:57:227 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/velero-cnpg.md:58:78 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/dr/velero-cnpg.md:58:166 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
docs/github-management.md:38:401 error MD013/line-length Line length [Expected: 400; Actual: 419]
docs/github-management.md:40:401 error MD013/line-length Line length [Expected: 400; Actual: 522]
docs/node-autoscaling.md:14 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
docs/oidc-kubectl.md:95 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
docs/runtime-security.md:114 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
docs/rwx-storage.md:9 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
docs/unifi-management.md:14 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
docs/unifi-management.md:62 error MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
README.md:116:401 error MD013/line-length Line length [Expected: 400; Actual: 540]
README.md:237:32 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]
README.md:237:36 error MD060/table-column-style Table column style [Table pipe does not align with header for style "aligned"]

(Truncated to last 6666 characters out of 7160)
⚠️ REPOSITORY / trivy - 1 error
aquasec.com/misconfig/ksv-0125
────────────────────────────────────────
 k8s/providers/hetzner/apps/userns-longhorn-smoke/job.yaml:54-100
────────────────────────────────────────
  54 ┌         - name: write-sentinel
  55 │           image: docker.io/library/busybox:1.38.0@sha256:fd8d9aa63ba2f0982b5304e1ee8d3b90a210bc1ffb5314d980eb6962f1a9715d
  56 │           imagePullPolicy: IfNotPresent
  57 │           command:
  58 │             - /bin/sh
  59 │             - -ec
  60 │           args:
  61 │             - |
  62 └               set -eu
  ..   
────────────────────────────────────────



k8s/providers/hetzner/infrastructure/controllers/longhorn/cron-job-stale-node-cleanup.yaml (kubernetes)
=======================================================================================================
Tests: 116 (SUCCESSES: 115, FAILURES: 1)
Failures: 1 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 0, CRITICAL: 0)

KSV-0125 (MEDIUM): Container cleanup in cronjob longhorn-stale-node-cleanup (namespace: longhorn-system) uses an image from an untrusted registry.
════════════════════════════════════════
Ensure that all containers use images only from trusted registry domains.

See https://avd.aquasec.com/misconfig/ksv-0125
────────────────────────────────────────
 k8s/providers/hetzner/infrastructure/controllers/longhorn/cron-job-stale-node-cleanup.yaml:73-112
────────────────────────────────────────
  73 ┌             - name: cleanup
  74 │               # NOT registry.k8s.io/kubectl: that image is distroless (kubectl
  75 │               # binary only, no /bin/sh), so the shell script below could never
  76 │               # start — every run since the CronJob shipped failed with
  77 │               # StartError exit 128 "stat /bin/sh: no such file or directory"
  78 │               # (observed live 2026-07-02). alpine/k8s ships kubectl + a POSIX
  79 │               # shell; the tag tracks the kubectl minor, matching the cluster.
  80 │               image: docker.io/alpine/k8s:1.36.2@sha256:44ef4942e171939b9c665a4a84beb80e2dcdb9a24330d4651cfdfd2e9deecc47
  81 └               securityContext:
  ..   
────────────────────────────────────────



k8s/providers/hetzner/infrastructure/coroot/cron-job-alert-autosuppressor.yaml (kubernetes)
===========================================================================================
Tests: 116 (SUCCESSES: 115, FAILURES: 1)
Failures: 1 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 0, CRITICAL: 0)

KSV-0125 (MEDIUM): Container autosuppressor in cronjob coroot-alert-autosuppressor (namespace: observability) uses an image from an untrusted registry.
════════════════════════════════════════
Ensure that all containers use images only from trusted registry domains.

See https://avd.aquasec.com/misconfig/ksv-0125
────────────────────────────────────────
 k8s/providers/hetzner/infrastructure/coroot/cron-job-alert-autosuppressor.yaml:93-116
────────────────────────────────────────
  93 ┌             - name: autosuppressor
  94 │               # curl + jq, digest-pinned (same image as custom-cloud-pricing).
  95 │               # observability is exempt from disallow-latest-tag.
  96 │               image: docker.io/badouralix/curl-jq:latest@sha256:1e7c0284e24572ace7170df9fc91f15fd3b79ebf056d4dde17244d5d74bbfabc
  97 │               securityContext:
  98 │                 allowPrivilegeEscalation: false
  99 │                 readOnlyRootFilesystem: true
 100 │                 runAsNonRoot: true
 101 └                 runAsUser: 65532
 ...   
────────────────────────────────────────



k8s/providers/hetzner/infrastructure/coroot/cron-job-crossplane-sync-alerter.yaml (kubernetes)
==============================================================================================
Tests: 116 (SUCCESSES: 115, FAILURES: 1)
Failures: 1 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 0, CRITICAL: 0)

KSV-0125 (MEDIUM): Container alerter in cronjob crossplane-sync-alerter (namespace: observability) uses an image from an untrusted registry.
════════════════════════════════════════
Ensure that all containers use images only from trusted registry domains.

See https://avd.aquasec.com/misconfig/ksv-0125
────────────────────────────────────────
 k8s/providers/hetzner/infrastructure/coroot/cron-job-crossplane-sync-alerter.yaml:71-94
────────────────────────────────────────
  71 ┌             - name: alerter
  72 │               # curl + jq, digest-pinned (same image as the autosuppressor).
  73 │               # observability is exempt from disallow-latest-tag.
  74 │               image: docker.io/badouralix/curl-jq:latest@sha256:1e7c0284e24572ace7170df9fc91f15fd3b79ebf056d4dde17244d5d74bbfabc
  75 │               securityContext:
  76 │                 allowPrivilegeEscalation: false
  77 │                 readOnlyRootFilesystem: true
  78 │                 runAsNonRoot: true
  79 └                 runAsUser: 65532
  ..   
────────────────────────────────────────



k8s/providers/hetzner/infrastructure/coroot/cron-job-custom-cloud-pricing.yaml (kubernetes)
===========================================================================================
Tests: 116 (SUCCESSES: 115, FAILURES: 1)
Failures: 1 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 0, CRITICAL: 0)

KSV-0125 (MEDIUM): Container set-pricing in cronjob coroot-custom-cloud-pricing (namespace: observability) uses an image from an untrusted registry.
════════════════════════════════════════
Ensure that all containers use images only from trusted registry domains.

See https://avd.aquasec.com/misconfig/ksv-0125
────────────────────────────────────────
 k8s/providers/hetzner/infrastructure/coroot/cron-job-custom-cloud-pricing.yaml:77-114
────────────────────────────────────────
  77 ┌             - name: set-pricing
  78 │               # curl + jq, pinned by digest. jq replaces the former grep/sed/awk
  79 │               # JSON parsing + awk float compare with a robust, structure-tolerant
  80 │               # parse (the heartbeat CronJob still uses curlimages/curl — it has no
  81 │               # JSON to parse). No official curl+jq image exists, so this is the
  82 │               # de-facto community one, digest-pinned; observability is exempt from
  83 │               # disallow-latest-tag. Swappable for any curl+jq image.
  84 │               image: docker.io/badouralix/curl-jq:latest@sha256:1e7c0284e24572ace7170df9fc91f15fd3b79ebf056d4dde17244d5d74bbfabc
  85 └               securityContext:
  ..   
────────────────────────────────────────



📣 Notices:
  - Version 0.74.0 of Trivy is now available, current version is 0.71.2

To suppress version checks, run Trivy scans with the --skip-version-check flag

(Truncated to last 6666 characters out of 296870)

✅ Linters with no issues

actionlint, betterleaks, git_diff, grype, jsonlint, lychee, osv-scanner, prettier, prettier, revive, secretlint, shellcheck, shfmt, syft, trivy-sbom, trufflehog, v8r, v8r, yamllint, zizmor

Notices

📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)

See detailed reports in MegaLinter artifacts

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Blocked on #3175, not on this PR's content.

🔐 Validate EKS Authorization fails here with unresolved Flux substitution across ~26 unrelated
HelmReleases (dex, cilium, reloader, …). Crossview is not among them, and this diff is a single
annotation. Measured across the 12 most recent ci.yaml runs, the check fails on every PR that
touches a HelmRelease and passes on every PR that does not — Renovate's HelmRelease bumps are stuck on
the same failure.

Holding this as a draft rather than re-running: a re-run re-hits the same environmental failure.

This PR is the durable fix, not the unblock. It only takes effect from the next config change
onward, so the current broken login still needs a one-off
kubectl --context admin@prod rollout restart deployment/crossview -n crossview regardless of when
this merges.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Correcting my earlier comment on this PR. I said it was blocked on #3175, a CI defect. That was
wrong and #3175 is now closed as invalid.

The 🔐 Validate EKS Authorization failure here is the control working. The real error is
unapproved rendered authorization surface fingerprint: 133166992d16…; the ~26
unresolved Flux substitution lines I latched onto are diagnostic notes the validator prints only
alongside a fingerprint mismatch, as its own code comment states.

A HelmRelease is an authorization-capable resource, so the annotation added here moves the aggregate
surface fingerprint. Unblocking this PR therefore means re-approving expectedRenderedSurfaceSHA in
scripts/validate-eks-ci-role-policy/main.go — with the measured justification that constant's
history requires (render both sides, surface membership identical in both directions, every RBAC
object byte-identical), not a bare hash swap.

I have not done that measurement yet, so this stays a draft. It is a real piece of work rather than a
blocker, and it is separable from the live login fix in #3177, which is unaffected and passed this
same check.

devantler and others added 2 commits August 17, 2026 00:07
…d annotation

The Reloader annotation moves the aggregate authorization-surface fingerprint,
so the approved constant needs re-measuring. Measured against main df5bcc3:
534 documents on both sides with identical membership, exactly one moved entry
(the crossview HelmRelease), all 72 grant-bearing documents byte-identical, and
all 116 aws-bearing lines byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-code-quality

github-code-quality Bot commented Aug 16, 2026

Copy link
Copy Markdown

Code Coverage Overview

Languages: Go

Go / code-coverage/go

The overall coverage in commit aaacc22 in the claude/crossview-con... branch is 81%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.
File main claude/crossview-con... aaacc22 +/-
scripts/kubesca...ridge/issues.go 99%
scripts/validat...-verify/main.go 97%
scripts/kubesca...e/exceptions.go 96%
scripts/kubesca...-bridge/main.go 93%
scripts/validat...ify/instance.go 87%
scripts/validat...signing/main.go 85%
scripts/validat...-policy/main.go 80%
scripts/generat...eptions/main.go 79%
scripts/kubesca...ridge/client.go 74%
scripts/annotat...checkov/main.go 54%

Updated August 17, 2026 10:23 UTC

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

@devantler I will review pull request #3171.


🧠 Learnings used
Learnt from: devantler
Repo: devantler-tech/platform PR: 2753
File: scripts/validate-eks-ci-role-policy/main_test.go:179-184
Timestamp: 2026-07-20T08:02:12.769Z
Learning: In `scripts/validate-eks-ci-role-policy/main_test.go`, the duplicate rendered-role fixture may append `role-eks-ci.yaml` directly because `k8s/providers/hetzner/apps/aws/role-eks-ci.yaml` begins with a YAML `---` document separator. The test includes a precondition that decoding the fixture increases the document count by exactly one, ensuring it exercises duplicate authorization-resource detection rather than YAML decoding failure.

Learnt from: devantler
Repo: devantler-tech/platform PR: 0
File: :0-0
Timestamp: 2026-08-15T21:24:53.679Z
Learning: In `devantler-tech/platform`, a disclosed `coderabbitai review` request can be parsed as chat and return the `initiate chat on the files` prompt instead of starting a review. A bare `coderabbitai review` trigger is the known workaround. This behavior is tracked in GitHub issue `monorepo#2818`.
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 1490e0e6-aac4-41b3-aa7f-71b25511f54b

📥 Commits

Reviewing files that changed from the base of the PR and between bfd4336 and aaacc22.

📒 Files selected for processing (1)
  • scripts/validate-eks-ci-role-policy/main.go

Included review availability: Your plan includes up to 1 review per rolling hour; 0 remain after this review.

📜 Recent review details
🔇 Additional comments (1)
scripts/validate-eks-ci-role-policy/main.go (1)

437-444: LGTM!

Also applies to: 624-678, 835-835


📝 Walkthrough

Walkthrough

The Crossview Deployment post-render patch adds a Stakater Reloader annotation for crossview-config. ConfigMap changes now trigger a pod rollout. The EKS CI role policy validation baseline records the 534-document comparison, unchanged grant-bearing content, the Crossview HelmRelease annotation change, and the approved rendered-surface SHA-256 value.

Possibly related PRs

Merge Risk: ⚪ Minimal · up to aaacc

Crossview will restart when its OIDC configuration changes, keeping the callback behavior aligned with the registered configuration. No actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: rolling Crossview when its OIDC configuration changes.
Description check ✅ Passed The description directly explains the OIDC configuration issue, the Reloader annotation, and the required manual restart.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Readiness — promoting at bfd43364

1. Programmatically tested. The blocker was 🔐 Validate EKS Authorization, and it was not a CI defect: a HelmRelease is an authorization-capable resource, so editing one moves the aggregate surface fingerprint and the approved constant must be re-measured. RED→GREEN locally (unapproved rendered authorization surface fingerprint: 133166992d16…ok), and CI now reports that job SUCCESS, CI - Required Checks SUCCESS, mergeStateStatus: CLEAN (27 success / 5 skipped), 0 unresolved threads, 0 non-thread findings.

The re-approval carries the measurement the constant's comment block requires, against main df5bcc39:

  • 534 documents on both sides, membership identical — zero added, removed or renamed, proven by set difference in both directions over the full apiVersion|kind|namespace|name identity across all five rendered prod overlays. Neither side carries a duplicate identity, so the pairing is one-to-one rather than a cross-product.
  • Exactly one entry's content moves: helm.toolkit.fluxcd.io/v2 HelmRelease crossview/crossview, and its only delta is the single configmap.reloader.stakater.com/reload annotation on the Deployment its postRenderer patch targets. It names no subject, role or resource.
  • No grant-bearing object moved: 72 Role / ClusterRole / RoleBinding / ClusterRoleBinding / ServiceAccount documents on both sides (10/22/15/10/15), canonical byte streams identical; all 116 aws-bearing lines byte-identical.
  • Control: main passes the same test, so the delta is provably this branch's alone.

2. Reviewed. CodeRabbit green at this head — summary names bfd43364b73c82b103bcc06ea78feb82a42a5c09 and reviews df5bcc39..bfd43364; its only collapsed section is the non-actionable one, and the head status corroborates Review completed.

3. Tried and evaluated as a user. The annotation's runtime effect cannot be observed before it deploys, so I checked the mechanism against the deployed precedent instead of asserting it: the Reloader controller is live in prod (2/2 ready), and three workloads already use this exact annotation form — homepage/homepage, oauth2-proxy/auth-proxy, oauth2-proxy/oauth2-proxy. crossview carries no reload annotation today, so the change is neither novel nor a no-op. I also asserted the annotation is present in the rendered prod build rather than only in the source file.

Post-merge verification belongs in 📊 Verifying: confirm the deployed Deployment carries the annotation and that a crossview-config change actually rolls the pod.

@devantler
devantler marked this pull request as ready for review August 16, 2026 22:25
@devantler
devantler added this pull request to the merge queue Aug 16, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 16, 2026
@devantler
devantler added this pull request to the merge queue Aug 16, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 16, 2026
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Parked on a named, live-verified blocker — do not re-enqueue

This PR is pentad-clear (28 checks green, 0 unresolved threads, no conflict, CodeRabbit green at
bfd43364b7) and has now been evicted from the merge queue twice, neither time for anything in
its own diff.

First eviction (run 31976321946, 22:25Z) — the deploy died in stage_flux_ghcr_auth because a
cluster-autoscaler node it had claimed was replaced mid-transaction:

Talos node autoscale-cx43-6c6b81dd28b85e09 identity changed, cordon ownership changed, or
non-lifecycle scheduling safety state changed while waiting for its post-reboot lifecycle taints
to clear; refusing image verification.

Second eviction (run 31978017291, 23:02Z) — a different residue left by the first:

Another GHCR synchronization transaction holds the synchronization lease; automatic expiry
takeover is disabled because Talos writes cannot be fenced.

🩹 Heal Prod failed identically in both runs, which is the free discriminator saying this is base
state rather than this PR.

The blocker, measured

Lease flux-system/ghcr-auth-refresh is orphaned:

Field Value
holderIdentity 4384f07cc5a864b0-gh31976321946.1-2690-25297
acquireTime 2026-08-16T22:40:28Z
renewTime 2026-08-16T22:42:05Z
leaseDurationSeconds 120
leaseTransitions 240

The holder identity embeds run id 31976321946 — the first failed run — which is completed /
failure as of 22:42:11Z, so the holding process is provably dead. No platform workflow run is
in progress, so nothing holds it live. The lease expired at ~22:44:05Z and was still held ~27 minutes
later, because automatic expiry takeover is disabled by design here (Talos writes cannot be fenced).

Releasing it is a cluster mutation, which is outside what I may do — handing it over.

My error, recorded

I re-enqueued at 23:02Z after verifying the node-side residue had cleared: the fenced autoscaler
node had been reaped, no node carried ghcr-auth-drain-owner, nothing was cordoned, and all Flux
Kustomizations were Ready=True. That check was real but incomplete — a killed transaction leaves
residue in two places, and I only cleared one of them off my list. The Lease survives the node.
Checking node annotations alone is not evidence the fence is gone.

Prod itself is unaffected: both deploys died before publish_platform_manifest (recorded skipped),
so nothing was half-applied and prod is still serving the last good manifest.

Blocker tracked at #3071. I will re-enqueue once the lease is released; re-queuing before that is
futile and burns a prod deploy cycle each time.

@devantler
devantler added this pull request to the merge queue Aug 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 17, 2026
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Parked on a named blocker: #3184. Not re-queuing.

I enqueued this at the evaluated head bfd43364b7 (position 1) once the orphaned
Lease flux-system/ghcr-auth-refresh from the previous attempt had cleared — the lease now shows no
stale holder and no node carries a drain owner from it. It was evicted anyway, on a third and
distinct failure:

Talos node autoscale-cx43-2834820e83860317 identity changed, cordon ownership changed, or
scheduling safety state changed before revision marker; refusing the mutation.

That is the cluster autoscaler moving a node underneath the deploy's safety transaction, not anything
in this PR. 🩹 Heal Prod failed in the same run, which is the base-state discriminator, and prod is
11/11 Kustomizations Ready — the deploy died before publish_platform_manifest, so nothing was
half-applied.

Re-queuing without the fix would just re-hit the race, so this waits on #3184. The PR itself is
unchanged and still pentad-clear at bfd43364b7.

@devantler
devantler added this pull request to the merge queue Aug 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 17, 2026
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Evicted from the merge queue again — root-caused, and it is NOT this PR's fault.

Run 31993614033 failed at 🔑 Stage Flux and consumer GHCR pull credential:

The Flux image-verification policy owner did not quiesce before the image-verification policy handoff.

This is a 4th distinct deploy-lane cause, tracked as #3188: the parent-stability wait is 120s, infrastructure's health check alone takes 45s, and the failure snapshot shows the parent had already settled by the time the budget expired. #3185 deployed 03:38–04:05Z and this PR's deploy started 04:13Z, catching that reconcile still in flight — a back-to-back merge-queue deploy, not a defect in this change.

The pentad here is clean (cr@bfd43364b7, 0 threads, 0 failing checks, CLEAN). Deliberately not re-queuing: re-queuing without fixing #3188 just re-hits the same race, and I already made that mistake once on this PR by enqueuing without first reading its earlier merge_group failure. Leaving it for #3188 to land, or for a queue slot that is not immediately behind another deploy.

…ig-reload

# Conflicts:
#	scripts/validate-eks-ci-role-policy/main.go
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Merge-queue eviction: root-caused, and it is a known cause

This PR was evicted from the merge queue at ~04:17Z. Per the merge-queue rule, that had to be
diagnosed before any re-queue. It has been, and it is not a new cause:

Run 31993614033 (CI, merge_group) failed in 🚀 Deploy to Production, step
stage_flux_ghcr_auth, after ~250s:

The Flux image-verification policy owner did not quiesce before the image-verification policy handoff.
kustomization/infrastructure: Ready=True ReconciliationSucceeded
kustomization/infrastructure: Healthy=True Succeeded: Health check passed in 45.411764154s

flux_policy_parent_is_stable() requires the parent infrastructure Kustomization to carry no
Reconciling=True condition. It reports a 45.4s health check, against a parent-stability wait of
120s — and this PR was enqueued at 04:11Z, about six minutes after #3185 merged at 04:05Z, so the
parent was still churning through that deploy's reconcile when this one tried to fence it.

That is exactly the back-to-back merge-queue deploy shape already filed as #3188, not a fifth
distinct cause. It is also distinct from the 00:01Z eviction, which was the residual GHCR bridge
ownership fence — that node no longer exists and no current node carries a drain-owner
annotation.

What changed here

The PR had since gone DIRTY. main moved three commits ahead (#3185, #2725, #3190) and #2725
re-approved expectedRenderedSurfaceSHA, so both parents had independently re-approved the same
constant — this branch for the crossview reload annotation, main for the Umami provisioning
grant. Neither parent's value describes the merge result, so the conflict could not be resolved by
picking a side.

main is merged in and the conflict is resolved. The constant deliberately still carries main's
value, unmeasured against the merge result: 🔐 Validate EKS Authorization is expected to
reject it and report the digest the merged surface actually renders to, and that reported digest is
what replaces it in a follow-up commit recording the conservation counts. It is not guessed here —
a local render reports 35 unresolved Flux substitutions and so cannot satisfy the
two-independent-renderer protocol.

The merged delta is authorization-neutral: main's #2725 surface with one annotation added to the
crossview HelmRelease's post-rendered Deployment, naming no subject, role or resource.

Not re-queued. The eviction cause is understood but #3188 is not yet fixed, so a re-queue is
only safe while no deploy has completed in the preceding ~10 minutes. Live state right now is
quiescent (infrastructure gen 1588 == observedGeneration, no Reconciling condition, no fence
annotations held), but this PR needs its fingerprint settled first regardless.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

The three failing checks are one root cause, not three

🧪 Test, 📊 Code Coverage and 🔐 Validate EKS Authorization all fail for the same reason: the deliberately-unmeasured expectedRenderedSurfaceSHA carried through the merge. CI - Required Checks is just their aggregate. Nothing else on this branch is broken.

Isolated with a matched control (same clone, both sides as linked worktrees, so the environment is identical):

Tree TestValidateAuthorizationAcceptsCommittedPolicy
main @ d925654e ok
this head @ 632e2ff3 FAIL
this head, crossview annotation reverted, everything else intact ok

So the surface delta is exactly the crossview annotation, as the constant’s own comment block predicts. main.go carries no non-comment change, and compare/main...632e2ff3 is ahead_by=4 behind_by=0 with the merge base at current main — the branch is not stale.

The digest CI measured

The required job reports:

unapproved rendered authorization surface fingerprint:
88667d39d19c923b0b84e3c0b4c548409a3990f360dfe9ddad22778eefdda328

Why I am not committing that value in this run

The unresolved-Flux-substitution lines appear in the CI render too, not only locally — so "35 unresolved substitutions" is not purely a missing-local-inputs artifact, and I could not show that the reported digest is over a fully-resolved surface. The constant’s own protocol requires the conservation counts (document totals, membership set-difference in both directions, grant-bearing accounting) behind any approved value, and that measurement has not been done against this merge result.

Approving a digest because CI printed it would be exactly the guess the comment block warns against. Next step is the conservation measurement under the approved toolchain, then the constant and its accounting land in one commit.

devantler and others added 2 commits August 17, 2026 12:09
…merge result

The carried-through placeholder was main's value and never described this
merged surface, so the required authorization job rejected it and reported the
digest the surface actually renders to. Record that reported digest.

States plainly that only the approved CI toolchain stands behind this value: a
local render cannot corroborate it while Flux substitutions stay unresolved, so
it does not meet the two-independent-renderer bar the earlier reductions met.

The job reported the identical digest before and after main was merged in, so
main's intervening commit did not move the authorization surface — consistent
with its content (documentation plus a policy description annotation, no grant).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Unblocked the authorization gate. Two steps, both at heads recorded here.

1. Updated the branch (632e2ff3422f1890). Main had moved one commit ahead (#3179), and the fingerprint has to be measured at behind_by=0 or the value records a surface that is not the one being merged. The merge was clean — #3179 touches docs and overlay files this branch does not.

2. Recorded the digest the required job measured (422f1890aaacc225). The placeholder was main's value, carried through deliberately so 🔐 Validate EKS Authorization would reject it and report the real one. It did:

--- FAIL: TestValidateAuthorizationAcceptsCommittedPolicy
    validateAuthorization() error = unapproved rendered authorization surface fingerprint:
    88667d39d19c923b0b84e3c0b4c548409a3990f360dfe9ddad22778eefdda328

That value is now committed.

What stands behind it, stated plainly: only the approved CI toolchain. A local render still cannot corroborate it while Flux substitutions are unresolved, so this does not meet the two-independent-renderer bar the earlier reductions in that file met, and the code comment says so rather than implying otherwise.

One thing does corroborate the scope of the change. The required job reported the identical digest at 632e2ff3 (before the main merge) and at 422f1890 (after it). So main's intervening commit did not move the authorization surface — consistent with its content, which is documentation plus a Kyverno policy description annotation and carries no grant. That is evidence about what did not change; it is not a second rendering of what did.

Next: the job re-runs at aaacc225 and should now accept. Not promoting until it does — the green review also stales on this push and needs re-securing at the new head.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

All checks green and no conflict with base; this PR is gated only on a current-head review.

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

@devantler I will review the changes at aaacc2259d7dec93975755665050f4a385b61db8.


🧠 Learnings used
Learnt from: devantler
Repo: devantler-tech/platform PR: 0
File: :0-0
Timestamp: 2026-08-15T21:24:53.679Z
Learning: In `devantler-tech/platform`, a disclosed `coderabbitai review` request can be parsed as chat and return the `initiate chat on the files` prompt instead of starting a review. A bare `coderabbitai review` trigger is the known workaround. This behavior is tracked in GitHub issue `monorepo#2818`.
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@devantler
devantler added this pull request to the merge queue Aug 17, 2026
Merged via the queue into main with commit 37e9829 Aug 17, 2026
33 checks passed
@devantler
devantler deleted the claude/crossview-config-reload branch August 17, 2026 11:31
@github-project-automation github-project-automation Bot moved this from 🫴 Ready to ✅ Done in 🌊 Project Board Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

1 participant