feat(accounts): opaque API keys resolved by hash, and a working key UI - #580
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Claude finished @alukach's task in 16s —— View job ✅ No blocking issues — safe to merge. I read the exchanges route, the keys table client, the server actions, the
Simplify (ponytail)
Docs
💰 Estimated review cost: $0.18 · 0m15s · 6 turns |
Merged
3 tasks done
3 tasks done
This was referenced Sep 25, 2026
Draft
alukach
force-pushed
the
feat/service-account-keys
branch
from
September 29, 2026 00:11
deed07c to
cd13694
Compare
This was referenced Sep 29, 2026
alukach
added a commit
to source-cooperative/data.source.coop
that referenced
this pull request
Sep 29, 2026
…he platform (#234) One commit on `main`, now that #232 (ADR-014) is merged. It revises ADR-013 in place, since ADR-013 was never implemented, and it replaces an earlier draft that added an ADR-015 and then folded it back. Part of source-cooperative/source.coop#491. Records the decision that replaces #233, developmentseed/multistore#147 and the proxy half of source-cooperative/source.coop#570. ## What it records **ADR-013, revised: API keys are opaque secrets resolved by the platform.** A key is `sck_` + 32 random bytes (a fixed 47 characters, `^sck_[A-Za-z0-9_-]{43}$`), stored as a sha256 hash on the key record in source.coop and shown once. Nothing signs it. At `/.sts` the proxy accepts it as `WebIdentityToken` from a POST body only, trims and format-checks it locally, hashes it, and asks `POST /api/v1/service-account-keys/exchanges` for `{account_id, key_id, active}`, cached 60s positive and negative, failing closed; then mints session credentials through the same code as every other exchange. The cache-miss path is rate-limited by client IP. A stock AWS SDK does the exchange and the refresh itself from `AWS_WEB_IDENTITY_TOKEN_FILE`; the emergency stop for a leaked key is disabling the service account. **Why the first Decision was withdrawn** is recorded under Context, each point checkable against #233 and source.coop#570: the revocation lookup was never optional, so the signature verified what the lookup restates; a Worker cannot fetch its own JWKS, so the "no new path" benefit was gone; minting was a source.coop→Ory→proxy→source.coop cycle whose `/.keys` would sign any `jti` for a manager; `exp` in the token overrode the editable record; and non-expiring keys died on the second rotation of a signing key shared with outbound federation. The ADR's original Context and its rejected alternatives stand. **Amendments**, in ADR-014's house form: ADR-005 gains the one proxy-as-itself route, with the sentinel subject `urn:source:data-proxy` that fails both account-id grammars; ADR-014's first two bullets amending ADR-013 lose their `sub`/`jti` wording. The RFC index gains ADR-014's row, which #232 omitted. It also fixes a sentence in ADR-014's "How it authenticates" section, which the automated review caught: it said a key's subject is the service account's id, and now says a key names its account on the key record. **Alternatives** rejected with reasons: the proxy-signed JWT (the first Decision), source.coop-signed JWTs verified via a source.coop JWKS, the two-hop exchange (opaque key → short-lived Ory ID token → `/.sts`), the proxy's `get_credential` slot, Ory-native tokens, and long-lived Ory refresh tokens. ## The two-hop spike the ADR cites Run on 2026-09-25 against the staging Ory project, driving the headless authorization-code flow source.coop already uses for a person's proxy credentials (`getOryIdToken` on `main`) with a service-account-shaped subject and, as a control, a random UUID that matches no identity. Both returned an RS256 ID token from `https://auth.staging.source.coop` whose `sub` was exactly the subject given, with the default 3600s lifetime. The run used a throwaway confidential OAuth2 client created and deleted for the purpose. Conclusion: Ory Network will mint for a subject with no identity record, so the two-hop design needs no proxy change and remains available; the ADR rejects it for the first release on the client-side cost to HPC and VM users, not on feasibility. ## Review Five targeted reviews of the plan and the decision text (security, proxy implementer, source.coop implementer, ops and client experience, ADR and issue hygiene), each returning "approve with changes"; all changes are folded in. The ones that moved a decision: keys refused in the query string (invocation logs are on; GDAL sends STS as a GET and is routed through the CLI); the sentinel is a URN, not a slug; unknown keys answer `active: false` rather than 404 so the existing cache code applies; rate limiting is per IP on misses and ships with the branch; the revocation numbers distinguish writes (60s) from restricted reads (300s) per the proxy's caches; ADR-005 is amended, not merely depended on. ## What follows #235, the proxy PR replacing #233 (its exchange branch survives; minting, self-verification and the multistore pin do not), source.coop#570 reworked in place, multistore#147 closed, #231 rewritten, and the epic's Phase 5 items edited. ## PR Checklist - [x] This PR has **no** breaking changes. (Documentation only.) - [x] I have updated or added new tests to cover the changes in this PR. (None apply; no code.) - [x] This PR affects the [Source Cooperative Frontend & API](https://github.com/source-cooperative/source.coop), and I have opened issue/PR source-cooperative/source.coop#570 to track the change. (Existing PR, to be reworked to this ADR.) ## Related Issues #230, #231, #232, #233, #235; developmentseed/multistore#146, developmentseed/multistore#147; source-cooperative/source.coop#491, source-cooperative/source.coop#548, source-cooperative/source.coop#561, source-cooperative/source.coop#570, source-cooperative/source.coop#580. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…y hash ADR-013 as revised (source-cooperative/data.source.coop#234): a key is `sck_` + 32 random bytes, stored only as its SHA-256, which is now the record's partition key; a random public `key_id` is the handle the UI, revoke and expiry actions use. Nothing signs a key and nothing calls the proxy to issue one, so the Ory ID-token round-trip, the compensating delete and `proxy-keys.ts` go, and `getOryIdToken` returns to private. The exchanges route becomes `POST /api/v1/service-account-keys/exchanges`, keyed by hash in the body. The proxy calls it before it knows which account a key belongs to, so it authenticates as itself: `verifyProxyAssertion`, extracted from `authenticateWithOidcToken`, checks the assertion, and the route accepts only the sentinel subject `urn:source:data-proxy`, which `authenticateWithOidcToken` now refuses outright so it can never become a session. Unknown, revoked, expired and disabled keys all answer `active: false` with no account, and a live key answers with its account and `key_id`; last use is recorded best-effort so a throttled write cannot refuse a live key. Pages strip the hash with `publicKey` before a record reaches the client component. The dialog's hint describes the stock-SDK setup, and the Storybook mock returns a key of the real shape. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The verify step now serves the API-key exchanges route directly as well as authenticateWithOidcToken, so its log lines carry their own operation tag; a proxy self-assertion failure is then separable from an end-user token failure when filtering logs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1eiTse4416N6uTgAy4Ddd
…t to Disable Opening the issue dialog threw: its "Never" expiry option had an empty value, which Radix Select refuses, so the dialog crashed as soon as it rendered. The expiry select is now `ApiKeyExpiryField`, which stands "never" for the empty `expires_in_days` the key actions read as no expiry; its stories are rendered by the smoke test, which skips the dialogs that contain it. The show-once view prints the five variables a stock AWS SDK or the AWS CLI needs to use the key from a file (`AWS_ROLE_ARN`, `AWS_WEB_IDENTITY_TOKEN_FILE`, `AWS_ENDPOINT_URL_STS`, `AWS_ENDPOINT_URL_S3`, `AWS_REGION`), filled in for this environment's proxy, with the same `role/FullAccess` ARN form and region as the GitHub workflow snippet; the dialog is 640px wide so the ARN line fits. `setApiKeyExpiry` had no caller: each live key now has a "Change expiry" dialog using the same field, starting at "never" for a key that never expires. The key list says to disable the account to stop every key at once, and the danger zone says what disabling does to credentials already issued and warns that enabling lets unrevoked keys sign in again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R1eiTse4416N6uTgAy4Ddd
alukach
force-pushed
the
feat/opaque-api-keys
branch
from
September 29, 2026 21:36
c85aff2 to
e703e7d
Compare
Contributor
Author
|
Rebased onto #570's current branch and force-pushed. #570 had been rebased onto a newer |
alukach
added a commit
that referenced
this pull request
Sep 29, 2026
#580) Into #570's branch, as the in-place rework that PR's description will need once this merges; #570 then no longer depends on the proxy and can land before it. Implements ADR-013 as revised in source-cooperative/data.source.coop#234. Part of #491 and #548. ## What **The key is opaque.** `sck_` + 32 random bytes in base64url, a fixed 47 characters matching `^sck_[A-Za-z0-9_-]{43}$`, from `randomBytes` (not the modulo-biased legacy generator). Its SHA-256 is the record's partition key; a random public `key_id` is what the UI, revoke and expiry actions handle. The table's `account_id` index stands. Nothing signs the key and nothing calls the proxy at issue: `proxy-keys.ts`, the Ory ID-token round-trip and the compensating delete are gone, and `getOryIdToken` is private again. **The proxy asks by hash, as itself.** `POST /api/v1/service-account-keys/exchanges` takes `{key_hash}` and answers `{account_id, key_id, active: true}` for a live key, or `{active: false}` for an unknown, revoked, expired or disabled one, with the reason only in the log under the forwarded `x-request-id`. Because the proxy does not yet know which account is calling, the route authenticates the proxy itself: `verifyProxyAssertion`, extracted from `authenticateWithOidcToken` and logging under its own operation name, verifies the assertion, and the route accepts only the sentinel subject `urn:source:data-proxy`. `authenticateWithOidcToken` refuses that subject outright, so it can never become a session, and the route never touches `getApiSession`, so no cookie reaches it. Last use is recorded best-effort: a throttled write must not refuse a live key. **The hash stays on the server.** `ServiceAccountKeyRecord` is the stored row; `publicKey` strips `key_hash` before either page hands records to the client component. The Storybook mock returns a key of the real shape. **The key UI works end to end.** Opening the issue dialog threw on #570's branch: its "Never" expiry option had an empty value, which Radix Select refuses. The select is now `ApiKeyExpiryField`, where "never" stands for the empty `expires_in_days` the actions read as no expiry. The smoke test renders its stories, and it skips the dialogs that contain it, which is how the crash went unnoticed. The show-once view prints the five variables a stock AWS SDK or the AWS CLI needs to use the key from a file, filled in for this environment's proxy, with the same `role/FullAccess` ARN form and region as the GitHub workflow snippet. `setApiKeyExpiry` had no caller: each live key now has a "Change expiry" dialog on the same field. The key list says to disable the account to stop every key at once, and the danger zone says what disabling does to credentials already issued and warns that enabling lets unrevoked keys sign in again. **Decisions to flag.** - `revokeApiKey` and `setApiKeyExpiry` find the key via `listByAccount(...).find(key_id)` rather than a third index: one query, and the ownership check comes free. - The route answers `active: false` rather than 404 for an unknown hash, so the proxy's existing cache code applies and the client learns nothing it did not already hold. - `after()` is not used for the last-use write; it is awaited in a `try/catch` because `after()` throws outside a request scope and would need mocking in every route test. - The printed `AWS_ROLE_ARN` names `FullAccess`, like the GitHub snippet already on `main`; both work once source-cooperative/data.source.coop#221 serves the named roles. Until then, `role/_default` is the name the proxy accepts. ## Stories - `IssueApiKeyDialog` › **Default**: https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-issueapikeydialog--default. Open it, pick "Never", and submit to reach the show-once view with the variables. - `ApiKeyExpiryField` › **Default**, **Never**: https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-apikeyexpiryfield--default - `ServiceAccountDetail` › **Default**, with "Change expiry" on each live key: https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-serviceaccountdetail--default - `ServiceAccountDetail` › **Disabled**, with the re-enable warning: https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-serviceaccountdetail--disabled - `ServiceAccountList` › **Default**: https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-serviceaccountlist--default Captured from a static Storybook build with headless Chromium, with no page errors:     ## Testing - `npx jest` on the touched suites, 5 suites and 204 tests, all passing: - `service-account-keys.test.ts`: stores only the hash and returns the key once; a different key each time; no-expiry; disabled account, non-manager, bad label and bad expiry refused before any write; revoke and expiry by `key_id`. - `oidc.test.ts`: the existing 16, plus: the sentinel never becomes a session even if an account had that id; `verifyProxyAssertion` returns claims without resolving anyone, and null for wrong issuer, wrong audience, expired, or no token. - The exchanges route: active with account and last-use written; still active when the write fails; revoked, expired, disabled and unknown all answer inactive without a write; 401 for any other subject or none; 400 for a non-hex body. - `ApiKeyExpiryField.test.tsx`: 90 days by default; an empty value for "never". - The stories smoke test, which now renders both `ApiKeyExpiryField` stories. - `npm run type-check`, `npm run lint` (one `require()` warning in the new route test, the same pattern as the trusts route test), `npm run build-storybook`: clean. - The screenshots above drive both dialogs in a real browser, including choosing "Never", which crashed before. ## Docs and ADRs data.source.coop: ADR-013 as revised in source-cooperative/data.source.coop#234 is what this implements. The route contract here (`{key_hash}` in, always-200 standing out, proxy-as-itself auth) is the seam that source-cooperative/data.source.coop#235 calls; source-cooperative/data.source.coop#235 is the proxy PR replacing source-cooperative/data.source.coop#233. The ADR-005 amendment in source-cooperative/data.source.coop#234 records the sentinel subject. docs.source.coop: the unattended-workflow guide (source-cooperative/docs.source.coop#34) carries the setup the show-once view prints, and uses the same variables. Nothing existing describes keys. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01R1eiTse4416N6uTgAy4Ddd --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
alukach
added a commit
that referenced
this pull request
Sep 29, 2026
…accounts (#570) Closes #548. Closes #546. API keys are the second Integration type beside the GitHub trusts #567 shipped, and #566 replaced proof of control with per-account trust. The `source.coop` half of #548. Part of #491. **Merge order:** this can merge and deploy before the proxy. It no longer calls the proxy; the proxy calls it. source-cooperative/data.source.coop#235 is the proxy half and needs the route here to be live, or every key exchange fails closed. ## What API keys for environments without OIDC — a server, a scheduler, an instrument — per ADR-013 as revised in source-cooperative/data.source.coop#234: **a key is an opaque secret that source.coop resolves by hash; nothing signs it.** Six commits on `main`: the original feature, dropping the Ory-id guard once #567 namespaced service-account ids, #580's rework to opaque keys, the key hint, a calmer key list, and that list as its own component with stories. **The key** is `sck_` + 32 random bytes in base64url: a fixed 47 characters, all entropy after the prefix, matching `sck_[A-Za-z0-9_-]{43}`, which is what gets registered with secret scanners (#561). It is shown once and never stored. **The record** (`service-account-keys` table) is keyed by the key's hex SHA-256, with a public `key_id` (a UUID) for listing, revoking and changing expiry, a label, who issued it and when, an optional expiry, `revoked_at` and `last_used_at`. `publicKey()` strips the hash before any record reaches a client component. **The hint**: the record also keeps the key's last four characters, and the key list shows each key as `sck_…Xy9Q`, so someone holding a key can tell which record, and so which service account, it is. Four characters are 24 of the key's 256 random bits, leaving far too many to guess. The hint only confirms a key in hand against its record; it is never used to look a key up, since keys across the platform will share it. The issue dialog says how the new key will be listed. Keys issued before the hint existed are listed without one. **Issuing** (`issueApiKey`): whoever manages the service account gives a label and an expiry (30/90/365 days, or never). The action generates the key, writes the record and returns the key once. A disabled service account is refused. **Revoking** sets `revoked_at`; **expiry** can be changed after issuance, including to never. **Exchanging**: `POST /api/v1/service-account-keys/exchanges` with `{key_hash}` is what the proxy calls at `/.sts` when a key is presented, authenticated as the proxy itself (`verifyProxyAssertion`, sentinel subject `urn:source:data-proxy`, recorded in the ADR-005 amendment in source-cooperative/data.source.coop#234). It always answers 200 with `{account_id, key_id, active}`. `active` means known, not revoked, not expired, and its service account not disabled; an unknown hash is answered as inactive, indistinguishable from a revoked one. It records last use. The proxy caches the answer for 60s, so revocation takes effect for new exchanges within that time; credentials already issued live to their session cap. **Resolving the subject**: after an exchange, the proxy's credentials name the service account itself, and `authenticateWithOidcToken` resolves it with `fetchByOryId`, then a service account by id. No service account's id can be someone's Ory identity id: #567 namespaces it as `{owner}--{id}`, and a UUID never contains `--`. **UI**: the service account's page (#567's `ServiceAccountDetail`) has an API keys section, rendered by `ApiKeyList`, a row per key. On the left, the label with a Revoked or Expired marker, and the key's hint beneath. On the right, two short lines: how it has been used ("Used 3 days ago", "Never used") and when it ends ("Expires in 5 months", "Never expires", "Revoked 9 months ago"). The exact dates, and who issued the key and when, are in their tooltip. Change expiry and Revoke are in a "⋯" menu; a revoked key has none, and keeps an invisible copy of the button so its lines align. The section header carries `IssueApiKeyDialog`, which shows the key once with a copy button, and the environment variables that point any AWS SDK or the AWS CLI at the proxy. The list row counts live keys as a way to sign in.  Every state a key can be in, from `ApiKeyList`'s Default story:  ## Stories - `ApiKeyList` › **Default** (every key state), **Single**, **WithoutHint**, **Empty** — https://source-coop-ui-git-feat-service-account-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-apikeylist--default (dates are set relative to today; hover a row's dates for the exact ones) - `ServiceAccountDetail` › **Default** — https://source-coop-ui-git-feat-service-account-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-serviceaccountdetail--default - `ServiceAccountList` › **Default** — https://source-coop-ui-git-feat-service-account-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-serviceaccountlist--default - `IssueApiKeyDialog` › **Default** — https://source-coop-ui-git-feat-service-account-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-issueapikeydialog--default (submitting reaches the show-once view with the hint line) - `ApiKeyExpiryField` — https://source-coop-ui-git-feat-service-account-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-apikeyexpiryfield--default ## Testing - `npx jest` — 79 suites, 835 tests, all pass on the branch as rebased onto `main`. `service-account-keys.test.ts` covers issuing (the record holds the hash, never the key; the hint is the key's last four characters; the key is returned once), no-expiry keys, refusals before any write, revoking only own keys, and expiry changes including to never. The exchanges route test covers the proxy-only auth, active and inactive answers, and last-use recording. - `npm run type-check`, `next lint`, `npm run build-storybook` — clean. ## Docs and ADRs data.source.coop: this implements ADR-013 as revised in source-cooperative/data.source.coop#234, with ADR-014 from source-cooperative/data.source.coop#232; the proxy side is source-cooperative/data.source.coop#235, which supersedes source-cooperative/data.source.coop#233. The hint is a display detail the ADR doesn't need. docs.source.coop: the unattended-workflow guide, source-cooperative/docs.source.coop#37 for source-cooperative/docs.source.coop#34, should mention matching a key to its account by its last four characters. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 29, 2026
alukach
added a commit
that referenced
this pull request
Sep 30, 2026
…ret scanning (#581) ## What I'm changing This adds two public routes that revoke a leaked service-account API key. It is the code half of #561. It builds on #596, merged, which ends every key in a CRC-32 checksum; #570 and #580's opaque `sck_` keys and `key_hash`-keyed records, both merged, are what these routes look up. - `POST /api/v1/service-account-keys/revocations` lets anyone who holds a key, its owner or a stranger who found it, revoke it without signing in, by sending `{"key": "sck_…"}`. It always answers 204, whether the key was live, already revoked or unknown. - `POST /api/v1/secret-scanning/github` is the endpoint for GitHub's secret-scanning partner program. It verifies GitHub's signature, revokes every reported token that `isApiKey` accepts (the pattern and the checksum), and answers with the feedback body the program defines. Both routes call one function, `revokeLeakedKey` in `src/lib/accounts/service-account-keys.ts`. It hashes the key, calls `fetchByHash`, revokes the key if it isn't revoked already, and logs `key_id` and `account_id`, never the key. `hashApiKey` moves there from `src/lib/actions/service-account-keys.ts`, which is a `use server` module and can't export it, so issuing a key and revoking one hash the same way. A key's record also says who revoked it. A new optional field, `revoked_via`, is `owner` when the key was revoked from settings, `holder` when someone presented it to the self-revoke route, and `github` when secret scanning reported it. A new table method, `serviceAccountKeysTable.revoke(key_hash, via)`, writes `revoked_via` and `revoked_at` in the same update, and all three ways of revoking a key call it; `set` no longer accepts `revoked_at`. The key list's tooltip adds who revoked the key after the date, for example "Revoked Sep 24, 2026 after GitHub found it in public". Keys revoked before this change have no `revoked_via`, and their tooltip is unchanged. ## How I did it **Self-revoke.** The key is read only from a JSON body. If `sck_` appears in the query string, the route answers 400 before any lookup, because request URLs are logged; this is the proxy's `/.sts` rule from ADR-013. The key is trimmed (a key copied out of a token file ends in a newline), hashed and looked up. The route never calls `getApiSession` and reads no cookies. A body without a well-formed key gets 400 rather than 204. The key format is public, so refusing a malformed key says nothing about any real key, and it tells someone who pasted the wrong thing that nothing was revoked. The 204 is the same for live, revoked and unknown keys, and presenting a live key revokes it, so a key can't be tested here without being killed. **GitHub endpoint.** This follows the [partner-program docs](https://docs.github.com/en/code-security/tutorials/secret-scanning-partner-program). - Before parsing anything, the route checks `Github-Public-Key-Signature` (a base64 DER ECDSA signature, P-256 with SHA-256) over the raw body bytes using `crypto.verify`. The public key is the one `Github-Public-Key-Identifier` names at `https://api.github.com/meta/public_keys/secret_scanning`. - Every failure to verify gets 401 and revokes nothing: a missing header, an unknown identifier, a bad signature, or a check that can't finish because the key fetch failed or a key or signature won't parse. The check is one function, `signedByGitHub`, which never throws and logs each refusal at warn with the key identifier. - GitHub's public keys are cached in module scope for the life of the function instance. GitHub rotates keys, so an identifier not in the cache triggers a refetch, but at most once every ten minutes. The keys endpoint allows 60 unauthenticated requests an hour per IP, Vercel's egress IPs are shared, and anyone can forge an identifier; without the throttle, a few dozen forged requests could use up the budget and stop an instance from ever verifying a real report. The attempt time is recorded before the fetch, so a failed or concurrent fetch counts against the throttle as well. - There is one cost. If GitHub starts signing with a newly published key within ten minutes of an instance's last fetch, that instance refuses the report with 401 until the window passes, so the report only gets through if GitHub sends it again. The partner-program docs don't say whether GitHub retries refused deliveries, so this is worth confirming during registration. - Tokens are selected by `isApiKey`, not by `type`. The type name is only fixed at registration, and the pattern is what gets registered. A token that fails the pattern or its checksum is never looked up and is left out of the answer, so a look-alike or a mangled key costs no database read. - The docs define a feedback body, and the route returns it: `[{token_raw, token_type, label}]`, with `true_positive` for a key whose record exists (whether it was revoked now or before) and `false_positive` otherwise. I chose `token_raw` over `token_hash` because GitHub already sent us the raw token, and the docs don't say how `token_hash` is encoded. The route answers 200; the docs don't specify status codes. - A signed body that isn't an array of `{token, type}` gets 400. `url` and `source` are only logged, so they aren't validated: an unexpected value in one of them shouldn't stop a batch of keys from being revoked. **What gets revoked.** An expired key is revoked too, and so is a key whose service account is disabled, because extending the expiry or enabling the account would bring the key back. An already-revoked key keeps its original `revoked_at`. Each revocation is stored with its `revoked_via` and logged at warn with the same `via`. The GitHub log line also carries the `url` and `source` where the key was found; they are logged, not stored. **Route paths.** `revocations` sits beside the existing `service-account-keys/exchanges`: a caller creates a revocation of a key in that collection, just as the proxy creates an exchange. The key can't go in the path because URLs are logged, and whoever finds a key doesn't know its public `key_id`. `secret-scanning/github` names the program and the partner, so an endpoint for another scanner, with its own signature scheme, can sit beside it. It isn't under `service-account-keys` because its body is GitHub's report format, not ours. **No config.** The keys URL belongs to GitHub and is the same in every environment, so it is a constant in the route rather than a `CONFIG` value. **Departure from the issue text.** #561 on GitHub still says to choose a marker in source-cooperative/data.source.coop#230, and that it blocks #548. Under the revised scope, the marker is fixed by ADR-013 (revised by source-cooperative/data.source.coop#234 and #242) as `^sck_[0-9A-Za-z]{36}$`, the last six characters a CRC-32 checksum of the rest, and #561 no longer blocks #548. This PR follows the revised scope. ## How you can test it On 581f0da, which adds `revoked_via`: `npm run type-check` passes, and `npx jest --forceExit src/app/api/v1/secret-scanning src/app/api/v1/service-account-keys src/lib/actions/service-account-keys.test.ts` passes (4 suites, 24 tests). Those suites now expect `revoke(hash, "github" | "holder" | "owner")`. I did not repeat the mutation pass or the random-order runs below. After the rebase onto `main` (d7fd522), type-check is clean and all 82 suites pass. Earlier, as of 9e9bd56 on #596's branch: - `npm run type-check` passes. - `npx jest src/types/service-account-key.test.ts src/lib/actions/service-account-keys.test.ts src/app/api/v1/secret-scanning src/app/api/v1/service-account-keys src/components/features/service-accounts --forceExit`: 6 suites and 37 tests pass. These are this PR's two suites, which now include a key whose checksum fails, and the suites #596 changed; the key actions import `hashApiKey` from the new module. - `npm run lint` reports nothing in the files this PR touches. - A hand mutation pass, run before the rebase onto #596 (dd9fc72): each of ten deliberate breaks makes at least one test fail. They are disabling signature verification, dropping the refetch throttle, never refetching for an unseen identifier, recording the fetch time only after a successful fetch, letting a failed check escape as a 500, loosening the token filter, dropping the query-string refusal, dropping the trim, rewriting an already-revoked key, and logging the key. - The GitHub route keeps its key cache between requests, so I also ran its suite in eight random orders (`--randomize`, seeds 1 to 8) and ran each of its seven tests on its own, on 9e9bd56. All passed. The GitHub suite signs requests with a throwaway P-256 key from `node:crypto` against a mocked key fetch, and it mocks `Date.now` to show the ten-minute throttle. It also replays the signed sample request from GitHub's docs with GitHub's published key, which pins the signature encoding to what GitHub actually sends. Not tested: a live delivery from GitHub, which needs the registration listed under follow-ups. To try the self-revoke route by hand against a local server or a preview: ``` curl -i -X POST <host>/api/v1/service-account-keys/revocations -H 'content-type: application/json' -d '{"key":"sck_…"}' ``` It answers 204, and the key's service-account page shows it as revoked. The proxy refuses new exchanges of the key once its 60-second cache expires. The same request with `?key=sck_…` in the URL answers 400. The only UI change is the tooltip text. The [ApiKeyList Default story](https://source-coop-ui-git-feat-api-key-leak-revocation-radiantearth.vercel.app/?path=/story/features-service-accounts-apikeylist--default) now includes a key revoked by an owner and a key revoked by GitHub; hover over each key's dates to see who revoked it. ## Docs and ADRs - ADR-013 as revised (source-cooperative/data.source.coop#234) already describes the self-revoke route: "the same hash lookup, body-only, a uniform response, the same rate limit". This PR implements all of that except the rate limit, which is the WAF follow-up below, so the ADR still holds. The GitHub endpoint follows from registering the ADR's marker. It accepts raw keys only in GitHub-signed reports and changes nothing the proxy decides, so I read it as within ADR-013 rather than a new decision. I also checked ADR-005 (authorization) and ADR-014 (service accounts); neither is affected. - docs.source.coop: no page on `main` covers API keys or revocation yet. The unattended workflow guide, source-cooperative/docs.source.coop#34, should say what to do when a key leaks: POST it to this route, or disable the service account. ## Follow-ups - **GitHub partner registration** is external and takes weeks. What to send GitHub, the steps, the questions to ask, and the equivalent steps for GitLab, gitleaks and TruffleHog are logged on #561: #561 (comment) - **Rate limiting.** source.coop has no rate-limit infrastructure, and 256-bit keys make guessing infeasible, so this PR adds no dependency. A Vercel WAF rate-limit rule on both paths would give the self-revoke route the limit ADR-013 describes, and would cap floods of either route. Forged GitHub key identifiers no longer need it, because the ten-minute throttle holds each instance to six key fetches an hour. If other tenants on Vercel's shared egress IPs use up GitHub's unauthenticated limit anyway, GitHub's docs suggest a personal access token with no scopes, or conditional requests. A token would be a new secret in `CONFIG`, so it is left out until it's needed. - **Telling the owner.** The app has no email or notification mechanism, so a revocation is only logged and shown in the key list. Telling the owner that their key leaked, and where (the GitHub `url` is in the log), is left for a follow-up. ## Related Part of #561 (epic #491). Builds on #570 as reworked by #580, both merged. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01R1eiTse4416N6uTgAy4Ddd --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
alukach
added a commit
to source-cooperative/data.source.coop
that referenced
this pull request
Sep 30, 2026
Supersedes #233. Implements the proxy half of API keys as recorded in the revised ADR-013 (#234, stacked on #232). Pairs with source-cooperative/source.coop#580 (into source-cooperative/source.coop#570's branch), which serves the route this calls. Closes #231. Part of source-cooperative/source.coop#491. ## What I'm changing A service account's API key is an opaque `sck_` secret, 30 random base62 characters and a six-character CRC-32 checksum of them (ADR-013, #242), that source.coop stores only as a SHA-256 hash. Nothing signs it. `/.sts` now accepts one as `WebIdentityToken` and resolves it by asking source.coop: - **Body only.** A key is read from the POST form body. One in the query string is refused before any lookup with `API key must be sent in the request body, not the URL (request id …)`, because Cloudflare logs request URLs. A JWT in the query string still goes to the STS route as before. - **Local checks first.** Trim surrounding whitespace, since every hand-made token file ends in a newline, then require exactly `sck_` + 36 base62 characters whose last six are the CRC-32 of the thirty before them (IEEE, as zlib computes it, written out in a few lines rather than a crate). Anything else with the `sck_` prefix is refused without a lookup. - **One lookup, cached.** `POST {SOURCE_API_URL}/api/v1/service-account-keys/exchanges` with `{"key_hash"}`, authenticated as the proxy itself (sentinel subject `urn:source:data-proxy`, the one route ADR-005 now lets the proxy call as itself). The route always answers 200: `{account_id, key_id, active: true}` or `{active: false}`. Both are cached for 60 seconds, so revocation takes effect within a minute and an unknown key costs one lookup a minute. A non-200 or network failure fails closed as a 500 `InternalError`, which SDKs retry, and caches nothing. - **One refusal for the key, and one for a mangled one.** A key that fails its shape or checksum was cut short or mistyped, and reads `InvalidIdentityToken: API key is malformed; check that it was copied whole (request id …)`; the format is public, so this reveals nothing, and it still counts against the rate limit. Unknown, revoked, expired and disabled all read `InvalidIdentityToken: API key was not accepted (request id …)`, with the id also in `x-amzn-requestid`. The id is in the message because SDKs show the user nothing else. The proxy logs one WARN line per refusal with the reason it knows (`malformed`, `inactive`, `query_string`, `rate_limited`) plus the key id and an 8-hex hash prefix; source.coop logs which of unknown, revoked, expired or disabled under the same request id. - **Minting.** Credentials for the named account under the `_default` role, sealed like every other session, with the same 900s floor, 3600s default and `STS_MAX_SESSION_DURATION_SECS` cap. `RoleArn`'s account segment is ignored, as for an Ory token. `ReadOnly`/`FullAccess` arrive with #221. - **Rate limit.** A new `KEY_EXCHANGE_LIMIT` ratelimit binding, 100 attempts a minute per client IP, in every wrangler config. A client exchanges about once a session, so a cluster behind one NAT stays far under it. A deployment without the binding logs an error and does not refuse traffic. **Decisions to flag** - **The limit applies to every `sck_` attempt, not only cache misses.** The cache sits inside the fetch helper, and at 100 a minute per IP the distinction makes no difference to legitimate traffic. ADR-013's wording is updated in #234 to match. - **No `<RequestId>` element in the STS error XML.** That lives in multistore-sts's `build_sts_error_response`; the header plus the message cover what SDKs surface, so no upstream change is needed now. - **`namespace_id`s `1001` (production), `1002` (staging) and `1003` (previews)** for the ratelimit binding. They only need to be unique within the Cloudflare account. - **CodeQL flags `key_hash` as weak password hashing (`src/keys.rs`), and it should be dismissed as a false positive.** The rule matches on the name: it takes the key for a password. A key is 256 random bits, so a slow hash or salt adds nothing, and the lookup is a key get, so there is no comparison to time. This is how GitHub stores its own tokens, and ADR-013 (#234) records it so nobody later "fixes" it to bcrypt. I have not dismissed the alert; that is the repo owner's call. - **Security Audit.** It failed here because `main`'s lockfile carried a rustls advisory (RUSTSEC-2026-0285). #238 fixed that on `main`, and this branch is rebased onto the fix. ## How I did it - `src/keys.rs` (new, wasm-free): `parse_api_key`, `looks_like_api_key`, `key_hash`, `KeyStanding`, `credentials_for`. - `src/lib.rs`: `api_key_exchange` runs after `ApiAuth` is built and before the router, and returns `None` for anything that isn't an `sck_` exchange so the STS route handles it unchanged. `exchange_api_key` does the lookup and minting; `key_refusal` builds the uniform refusal; `finish` adds CORS and the request-id headers to a pre-gateway response; `within_rate_limit` wraps the binding. - `src/source_api/auth.rs`: `PROXY_SELF_SUBJECT`, `ApiCaller` (anonymous, an account, or the proxy), `authorization_header_as_self`; `authorization_header` refuses the sentinel so no request can claim it. - `src/source_api/cache.rs`: `get_or_fetch_key_standing`; `cached_fetch` takes a method, an optional JSON body and an `ApiCaller`. The cache key is `{api_url}?key_hash={hash}`, so it is a URL, as the Cache API requires, and is scoped to the environment's API. - `src/sts.rs`: `default_role` factored out of `StsCredentialRegistry::new`. - `wrangler.toml`, `wrangler.preview.toml`: the binding per environment. `README.md`: a Bindings table and an API keys section. From #233 this keeps `finish`, the shape of `api_key_exchange`/`exchange_api_key`, `credentials_for` and the method argument to `cached_fetch`. It drops `/.keys`, minting, self-verification against the proxy's own JWKS, the `api_key` role, `chrono`/`rsa` and the `[patch.crates-io]` pin on multistore; developmentseed/multistore#147 is not needed. ## How to test it - `cargo test`: all suites, including the new `tests/keys.rs` (a key's shape, trimming `\n` and `\r\n`, rejection of short, long, wrong-case, mistyped, wrong-checksum, non-base62, checksum-less 47-character and JWT tokens, checksum vectors computed independently with Python's zlib (a CRC above 2^31, one with a leading zero), assembled with `concat!` so scanners don't flag the file, the SHA-256 test vector, and credentials sealed for the account within floor, default and cap). Run by the pre-commit hook, along with `cargo clippy --target wasm32-unknown-unknown -- -D warnings` and `cargo check --target wasm32-unknown-unknown`. - `pytest tests/test_api_keys.py` against `wrangler dev` and `tests/stub_api.py`, which gains the exchanges route keyed by the hash of fixed test keys, with a per-hash call counter. Nine tests, all passing locally with wrangler 3.114: a live key exchanges; **an unmodified boto3, configured only by `AWS_ROLE_ARN`, `AWS_WEB_IDENTITY_TOKEN_FILE` (a file holding the key and a trailing newline) and `AWS_ENDPOINT_URL_STS`, acquires credentials**; the second exchange within 60s never reaches the API; a trailing newline is harmless; unknown and revoked keys get byte-identical refusals apart from the id, and the refusal is cached; a key in the query string is refused without a lookup; malformed keys, including a mistyped one whose checksum fails, are refused without a lookup and with the malformed message; a wrong role is reported as such; an API 500 fails closed and is not cached. `test_control_plane.py` and `test_writes.py` still pass; their credentialed tests need CI's GitHub token and were skipped locally. The checksum commits (d6745e0, 30ad22f) were run by CI's Integration Tests job, which exchanges the new-format keys against the worker; it passed on both. - End to end, once source.coop#580 is on a deployment this preview points at: issue a key from a service account's page, save it to a file, then ```sh AWS_WEB_IDENTITY_TOKEN_FILE=./key AWS_ROLE_ARN=arn:aws:iam::000000000000:role/_default \ AWS_ENDPOINT_URL_STS=https://<preview>/.sts AWS_ENDPOINT_URL_S3=https://<preview> AWS_REGION=us-east-1 \ aws s3 ls s3://<owner>/<product>/ ``` Revoke the key and see the next exchange refused within 60s, then quote the printed request id to find the proxy's and source.coop's log lines. Not run here: it needs source-cooperative/source.coop#580 deployed. ## PR Checklist - [x] This PR has **no** breaking changes. (JWT exchanges at `/.sts` are unchanged; the new binding is additive.) - [x] I have updated or added new tests to cover the changes in this PR. - [x] This PR affects the [Source Cooperative Frontend & API](https://github.com/source-cooperative/source.coop), and I have opened issue/PR source-cooperative/source.coop#580 to track the change. ## Related Issues Closes #231. Supersedes #233. ADR: #234 (revises ADR-013, amends ADR-005). Route: source-cooperative/source.coop#580, into source-cooperative/source.coop#570. Epic: source-cooperative/source.coop#491. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01R1eiTse4416N6uTgAy4Ddd --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Into #570's branch, as the in-place rework that PR's description will need once this merges; #570 then no longer depends on the proxy and can land before it. Implements ADR-013 as revised in source-cooperative/data.source.coop#234. Part of #491 and #548.
What
The key is opaque.
sck_+ 32 random bytes in base64url, a fixed 47 characters matching^sck_[A-Za-z0-9_-]{43}$, fromrandomBytes(not the modulo-biased legacy generator). Its SHA-256 is the record's partition key; a random publickey_idis what the UI, revoke and expiry actions handle. The table'saccount_idindex stands. Nothing signs the key and nothing calls the proxy at issue:proxy-keys.ts, the Ory ID-token round-trip and the compensating delete are gone, andgetOryIdTokenis private again.The proxy asks by hash, as itself.
POST /api/v1/service-account-keys/exchangestakes{key_hash}and answers{account_id, key_id, active: true}for a live key, or{active: false}for an unknown, revoked, expired or disabled one, with the reason only in the log under the forwardedx-request-id. Because the proxy does not yet know which account is calling, the route authenticates the proxy itself:verifyProxyAssertion, extracted fromauthenticateWithOidcTokenand logging under its own operation name, verifies the assertion, and the route accepts only the sentinel subjecturn:source:data-proxy.authenticateWithOidcTokenrefuses that subject outright, so it can never become a session, and the route never touchesgetApiSession, so no cookie reaches it. Last use is recorded best-effort: a throttled write must not refuse a live key.The hash stays on the server.
ServiceAccountKeyRecordis the stored row;publicKeystripskey_hashbefore either page hands records to the client component. The Storybook mock returns a key of the real shape.The key UI works end to end. Opening the issue dialog threw on #570's branch: its "Never" expiry option had an empty value, which Radix Select refuses. The select is now
ApiKeyExpiryField, where "never" stands for the emptyexpires_in_daysthe actions read as no expiry. The smoke test renders its stories, and it skips the dialogs that contain it, which is how the crash went unnoticed. The show-once view prints the five variables a stock AWS SDK or the AWS CLI needs to use the key from a file, filled in for this environment's proxy, with the samerole/FullAccessARN form and region as the GitHub workflow snippet.setApiKeyExpiryhad no caller: each live key now has a "Change expiry" dialog on the same field. The key list says to disable the account to stop every key at once, and the danger zone says what disabling does to credentials already issued and warns that enabling lets unrevoked keys sign in again.Decisions to flag.
revokeApiKeyandsetApiKeyExpiryfind the key vialistByAccount(...).find(key_id)rather than a third index: one query, and the ownership check comes free.active: falserather than 404 for an unknown hash, so the proxy's existing cache code applies and the client learns nothing it did not already hold.after()is not used for the last-use write; it is awaited in atry/catchbecauseafter()throws outside a request scope and would need mocking in every route test.AWS_ROLE_ARNnamesFullAccess, like the GitHub snippet already onmain; both work once Add the ReadOnly Role alongside FullAccess data.source.coop#221 serves the named roles. Until then,role/_defaultis the name the proxy accepts.Stories
IssueApiKeyDialog› Default: https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-issueapikeydialog--default. Open it, pick "Never", and submit to reach the show-once view with the variables.ApiKeyExpiryField› Default, Never: https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-apikeyexpiryfield--defaultServiceAccountDetail› Default, with "Change expiry" on each live key: https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-serviceaccountdetail--defaultServiceAccountDetail› Disabled, with the re-enable warning: https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-serviceaccountdetail--disabledServiceAccountList› Default: https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-serviceaccountlist--defaultCaptured from a static Storybook build with headless Chromium, with no page errors:
Testing
npx jeston the touched suites, 5 suites and 204 tests, all passing:service-account-keys.test.ts: stores only the hash and returns the key once; a different key each time; no-expiry; disabled account, non-manager, bad label and bad expiry refused before any write; revoke and expiry bykey_id.oidc.test.ts: the existing 16, plus: the sentinel never becomes a session even if an account had that id;verifyProxyAssertionreturns claims without resolving anyone, and null for wrong issuer, wrong audience, expired, or no token.ApiKeyExpiryField.test.tsx: 90 days by default; an empty value for "never".ApiKeyExpiryFieldstories.npm run type-check,npm run lint(onerequire()warning in the new route test, the same pattern as the trusts route test),npm run build-storybook: clean.Docs and ADRs
data.source.coop: ADR-013 as revised in source-cooperative/data.source.coop#234 is what this implements. The route contract here (
{key_hash}in, always-200 standing out, proxy-as-itself auth) is the seam that source-cooperative/data.source.coop#235 calls; source-cooperative/data.source.coop#235 is the proxy PR replacing source-cooperative/data.source.coop#233. The ADR-005 amendment in source-cooperative/data.source.coop#234 records the sentinel subject.docs.source.coop: the unattended-workflow guide (source-cooperative/docs.source.coop#34) carries the setup the show-once view prints, and uses the same variables. Nothing existing describes keys.
🤖 Generated with Claude Code
https://claude.ai/code/session_01R1eiTse4416N6uTgAy4Ddd