Skip to content

feat(accounts): issue, revoke and expire opaque API keys for service accounts - #570

Merged
alukach merged 6 commits into
mainfrom
feat/service-account-keys
Sep 29, 2026
Merged

alukach merged 6 commits into
mainfrom
feat/service-account-keys

Conversation

@alukach

@alukach alukach commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Closes #548. Closes #546. API keys are the second Integration type beside the GitHub trusts #567 shipped, and #566 replaced proof of control with per-account trust. The source.coop half of #548. Part of #491.

Merge order: this can merge and deploy before the proxy. It no longer calls the proxy; the proxy calls it. source-cooperative/data.source.coop#235 is the proxy half and needs the route here to be live, or every key exchange fails closed.

What

API keys for environments without OIDC — a server, a scheduler, an instrument — per ADR-013 as revised in source-cooperative/data.source.coop#234: a key is an opaque secret that source.coop resolves by hash; nothing signs it. Six commits on main: the original feature, dropping the Ory-id guard once #567 namespaced service-account ids, #580's rework to opaque keys, the key hint, a calmer key list, and that list as its own component with stories.

The key is sck_ + 32 random bytes in base64url: a fixed 47 characters, all entropy after the prefix, matching sck_[A-Za-z0-9_-]{43}, which is what gets registered with secret scanners (#561). It is shown once and never stored.

The record (service-account-keys table) is keyed by the key's hex SHA-256, with a public key_id (a UUID) for listing, revoking and changing expiry, a label, who issued it and when, an optional expiry, revoked_at and last_used_at. publicKey() strips the hash before any record reaches a client component.

The hint: the record also keeps the key's last four characters, and the key list shows each key as sck_…Xy9Q, so someone holding a key can tell which record, and so which service account, it is. Four characters are 24 of the key's 256 random bits, leaving far too many to guess. The hint only confirms a key in hand against its record; it is never used to look a key up, since keys across the platform will share it. The issue dialog says how the new key will be listed. Keys issued before the hint existed are listed without one.

Issuing (issueApiKey): whoever manages the service account gives a label and an expiry (30/90/365 days, or never). The action generates the key, writes the record and returns the key once. A disabled service account is refused. Revoking sets revoked_at; expiry can be changed after issuance, including to never.

Exchanging: POST /api/v1/service-account-keys/exchanges with {key_hash} is what the proxy calls at /.sts when a key is presented, authenticated as the proxy itself (verifyProxyAssertion, sentinel subject urn:source:data-proxy, recorded in the ADR-005 amendment in source-cooperative/data.source.coop#234). It always answers 200 with {account_id, key_id, active}. active means known, not revoked, not expired, and its service account not disabled; an unknown hash is answered as inactive, indistinguishable from a revoked one. It records last use. The proxy caches the answer for 60s, so revocation takes effect for new exchanges within that time; credentials already issued live to their session cap.

Resolving the subject: after an exchange, the proxy's credentials name the service account itself, and authenticateWithOidcToken resolves it with fetchByOryId, then a service account by id. No service account's id can be someone's Ory identity id: #567 namespaces it as {owner}--{id}, and a UUID never contains --.

UI: the service account's page (#567's ServiceAccountDetail) has an API keys section, rendered by ApiKeyList, a row per key. On the left, the label with a Revoked or Expired marker, and the key's hint beneath. On the right, two short lines: how it has been used ("Used 3 days ago", "Never used") and when it ends ("Expires in 5 months", "Never expires", "Revoked 9 months ago"). The exact dates, and who issued the key and when, are in their tooltip. Change expiry and Revoke are in a "⋯" menu; a revoked key has none, and keeps an invisible copy of the button so its lines align. The section header carries IssueApiKeyDialog, which shows the key once with a copy button, and the environment variables that point any AWS SDK or the AWS CLI at the proxy. The list row counts live keys as a way to sign in.

The API keys section: HPC cron job over sck_…Xy9Q, with Used 6 months ago and Expires in 5 months on the right and a ⋯ menu; Old laptop, marked REVOKED, over sck_…a_7k, with Never used and Revoked 9 months ago

Every state a key can be in, from ApiKeyList's Default story:

Five keys: HPC cron job, used 3 days ago, expires in 5 months; Instrument uploader, used today, never expires; Laptop, for testing, never used, expires next month; Last year's sync marked EXPIRED, used last month, expired last month; Old laptop marked REVOKED, never used, revoked 9 months ago. Each shows its sck_… hint; live keys have a ⋯ menu

Stories

Testing

  • npx jest — 79 suites, 835 tests, all pass on the branch as rebased onto main. service-account-keys.test.ts covers issuing (the record holds the hash, never the key; the hint is the key's last four characters; the key is returned once), no-expiry keys, refusals before any write, revoking only own keys, and expiry changes including to never. The exchanges route test covers the proxy-only auth, active and inactive answers, and last-use recording.
  • npm run type-check, next lint, npm run build-storybook — clean.

Docs and ADRs

data.source.coop: this implements ADR-013 as revised in source-cooperative/data.source.coop#234, with ADR-014 from source-cooperative/data.source.coop#232; the proxy side is source-cooperative/data.source.coop#235, which supersedes source-cooperative/data.source.coop#233. The hint is a display detail the ADR doesn't need. docs.source.coop: the unattended-workflow guide, source-cooperative/docs.source.coop#37 for source-cooperative/docs.source.coop#34, should mention matching a key to its account by its last four characters.

🤖 Generated with Claude Code

@alukach alukach added the blocked Waits on another PR merging first label Sep 22, 2026
@vercel

vercel Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
source-coop-ui Ready Ready Preview Sep 29, 2026 10:40pm UTC
source-cooperative Ready Ready Preview Sep 29, 2026 10:40pm UTC

Request Review

@claude

claude Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @alukach's task in 19s —— View job


✅ No blocking issues — safe to merge.

I read the actions, the exchanges route, the keys table client, the key types and the oidc.ts changes. I did not run the test suite.

  • Key handling: Only the hash is stored, and the key is returned once. The 32 random bytes, the write conditioned on attribute_not_exists, and the hex-only key_hash regex are all correct.
  • Exchanges route: It accepts only a proxy-signed assertion whose subject is PROXY_SELF_SUBJECT. authenticateWithOidcToken rejects that same subject, so a proxy assertion can't become a session. Unknown, revoked, expired and disabled keys all get the same {active:false} answer.
  • Concurrent writes: set updates one field and requires the record to exist. A revocation can't be overwritten by a last_used_at write, and a deleted key can't be brought back.
  • Read cache: cachedSend memoizes per request, so ownKey doesn't read stale key lists.

Simplify (ponytail)

  • src/lib/api/oidc.ts:~88: verifyProxyAssertion also logs a "No Bearer token" line under the new operation name. This is harmless, but it is the same debug noise on the API-key route.
  • src/app/api/v1/service-account-keys/exchanges/route.ts:56: the nested ternary that builds reason could be a small early-return helper. It reads fine as it is.
  • src/lib/actions/service-account-keys.ts:86: ownKey lists every key of the account to find one. That is fine at this scale. If accounts ever hold many keys, replace it with a get by key_id.

Docs

The description covers this. It names ADR-013 as revised in data.source.coop#234, ADR-014 in #232, the proxy PR #235, and docs.source.coop#37/#34, and it says why the hint needs no ADR change. Nothing is missing.


💰 Estimated review cost: $0.19 · 0m18s · 10 turns

@alukach

alukach commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

Pushed f93dcac for the review finding: the keys table now sets one field at a time with an UpdateCommand conditioned on the record existing (set(jti, field, value)), and the exchanges route, revoke and expiry each write only the field they own. Title and description still hold.

@alukach

alukach commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

Pushed a follow-up so the comparisons this branch added ask isServiceAccount (the guard now lives in src/types/account.ts, from #563). Description still holds.

@alukach

alukach commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

Stack #571 rebased to carry #565's no-backfill change (Ory identities stay on identity_id; the resolver's Ory-first chain is unchanged), plus one guard from the review: issueApiKey refuses a service account whose id is also someone's Ory identity id, since the proxy forwards the subject bare and the person would win. Description updated.

@alukach

alukach commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto #567's redesign. The API keys section moved from the old service-account card to the account's own page, ServiceAccountDetail. The list row counts live keys, and the key actions revalidate the page. Description and screenshot updated.

@alukach

alukach commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto #567's namespaced ids. New commit 009e255 drops the refusal of a key for an account whose id is also an Ory identity id: a namespaced id always contains -- and a UUID never does. The keys tests and stories use namespaced ids. Description updated.

@alukach

alukach commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main after #567's squash merge and force-pushed: the branch is now this PR's two commits only. The content is unchanged; the tree matches the previous head exactly. Description updated: #567 no longer blocks this, source-cooperative/data.source.coop#233 still does.

alukach and others added 4 commits September 29, 2026 15:25
API keys for environments without OIDC, per ADR-013 as amended by ADR-014: a JWT the data proxy signs, sub the service account, jti the record's id, exp optional, handed out once as sck_ plus the JWT. Issuing writes the record and asks the proxy to sign; a refused signature removes the record. Revocation and expiry each write one field. The proxy checks a key's standing on every exchange through the exchanges route, as the account. A key's subject is the account's own id, which the API resolves directly, so nothing else is written for it; an account whose id is also someone's Ory identity id, or that is disabled, gets no key. Rebuilt on the trust model in one commit; the earlier history is on GitHub in the PR.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REZWKgQy2PDETn6j9YpM4z
A key's subject is the service account's id, and the guard refused an account whose id was also someone's Ory identity id, since the resolver tries Ory first. Service account ids are now `{owner}--{id}`, and an Ory identity id is a UUID, which never contains `--`, so no service account can be refused by it. The keys tests and stories use namespaced ids.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REZWKgQy2PDETn6j9YpM4z
#580)

Into #570's branch, as the in-place rework that PR's description will
need once this merges; #570 then no longer depends on the proxy and can
land before it. Implements ADR-013 as revised in
source-cooperative/data.source.coop#234. Part of #491 and #548.

## What

**The key is opaque.** `sck_` + 32 random bytes in base64url, a fixed 47
characters matching `^sck_[A-Za-z0-9_-]{43}$`, from `randomBytes` (not
the modulo-biased legacy generator). Its SHA-256 is the record's
partition key; a random public `key_id` is what the UI, revoke and
expiry actions handle. The table's `account_id` index stands. Nothing
signs the key and nothing calls the proxy at issue: `proxy-keys.ts`, the
Ory ID-token round-trip and the compensating delete are gone, and
`getOryIdToken` is private again.

**The proxy asks by hash, as itself.** `POST
/api/v1/service-account-keys/exchanges` takes `{key_hash}` and answers
`{account_id, key_id, active: true}` for a live key, or `{active:
false}` for an unknown, revoked, expired or disabled one, with the
reason only in the log under the forwarded `x-request-id`. Because the
proxy does not yet know which account is calling, the route
authenticates the proxy itself: `verifyProxyAssertion`, extracted from
`authenticateWithOidcToken` and logging under its own operation name,
verifies the assertion, and the route accepts only the sentinel subject
`urn:source:data-proxy`. `authenticateWithOidcToken` refuses that
subject outright, so it can never become a session, and the route never
touches `getApiSession`, so no cookie reaches it. Last use is recorded
best-effort: a throttled write must not refuse a live key.

**The hash stays on the server.** `ServiceAccountKeyRecord` is the
stored row; `publicKey` strips `key_hash` before either page hands
records to the client component. The Storybook mock returns a key of the
real shape.

**The key UI works end to end.** Opening the issue dialog threw on
#570's branch: its "Never" expiry option had an empty value, which Radix
Select refuses. The select is now `ApiKeyExpiryField`, where "never"
stands for the empty `expires_in_days` the actions read as no expiry.
The smoke test renders its stories, and it skips the dialogs that
contain it, which is how the crash went unnoticed. The show-once view
prints the five variables a stock AWS SDK or the AWS CLI needs to use
the key from a file, filled in for this environment's proxy, with the
same `role/FullAccess` ARN form and region as the GitHub workflow
snippet. `setApiKeyExpiry` had no caller: each live key now has a
"Change expiry" dialog on the same field. The key list says to disable
the account to stop every key at once, and the danger zone says what
disabling does to credentials already issued and warns that enabling
lets unrevoked keys sign in again.

**Decisions to flag.**
- `revokeApiKey` and `setApiKeyExpiry` find the key via
`listByAccount(...).find(key_id)` rather than a third index: one query,
and the ownership check comes free.
- The route answers `active: false` rather than 404 for an unknown hash,
so the proxy's existing cache code applies and the client learns nothing
it did not already hold.
- `after()` is not used for the last-use write; it is awaited in a
`try/catch` because `after()` throws outside a request scope and would
need mocking in every route test.
- The printed `AWS_ROLE_ARN` names `FullAccess`, like the GitHub snippet
already on `main`; both work once
source-cooperative/data.source.coop#221 serves the named roles. Until
then, `role/_default` is the name the proxy accepts.

## Stories

- `IssueApiKeyDialog` › **Default**:
https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-issueapikeydialog--default.
Open it, pick "Never", and submit to reach the show-once view with the
variables.
- `ApiKeyExpiryField` › **Default**, **Never**:
https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-apikeyexpiryfield--default
- `ServiceAccountDetail` › **Default**, with "Change expiry" on each
live key:
https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-serviceaccountdetail--default
- `ServiceAccountDetail` › **Disabled**, with the re-enable warning:
https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-serviceaccountdetail--disabled
- `ServiceAccountList` › **Default**:
https://source-coop-ui-git-feat-opaque-api-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-serviceaccountlist--default

Captured from a static Storybook build with headless Chromium, with no
page errors:

![The issue dialog with a label and "Never — until revoked"
chosen](https://raw.githubusercontent.com/source-cooperative/source.coop/assets/opaque-api-keys/issue-api-key-form.png)

![The show-once view: the key, and five export lines for AWS_ROLE_ARN,
AWS_WEB_IDENTITY_TOKEN_FILE, AWS_ENDPOINT_URL_STS, AWS_ENDPOINT_URL_S3
and AWS_REGION with a copy
button](https://raw.githubusercontent.com/source-cooperative/source.coop/assets/opaque-api-keys/issue-api-key-show-once.png)

![The change-expiry dialog: "When should HPC cron job expire?" with the
expiry select, Close and
Save](https://raw.githubusercontent.com/source-cooperative/source.coop/assets/opaque-api-keys/change-expiry.png)

![A disabled service account: the API keys section with Change expiry
and Revoke on the live key, and the danger zone warning that enabling
lets unrevoked keys sign in
again](https://raw.githubusercontent.com/source-cooperative/source.coop/assets/opaque-api-keys/detail-disabled.png)

## Testing

- `npx jest` on the touched suites, 5 suites and 204 tests, all passing:
- `service-account-keys.test.ts`: stores only the hash and returns the
key once; a different key each time; no-expiry; disabled account,
non-manager, bad label and bad expiry refused before any write; revoke
and expiry by `key_id`.
- `oidc.test.ts`: the existing 16, plus: the sentinel never becomes a
session even if an account had that id; `verifyProxyAssertion` returns
claims without resolving anyone, and null for wrong issuer, wrong
audience, expired, or no token.
- The exchanges route: active with account and last-use written; still
active when the write fails; revoked, expired, disabled and unknown all
answer inactive without a write; 401 for any other subject or none; 400
for a non-hex body.
- `ApiKeyExpiryField.test.tsx`: 90 days by default; an empty value for
"never".
- The stories smoke test, which now renders both `ApiKeyExpiryField`
stories.
- `npm run type-check`, `npm run lint` (one `require()` warning in the
new route test, the same pattern as the trusts route test), `npm run
build-storybook`: clean.
- The screenshots above drive both dialogs in a real browser, including
choosing "Never", which crashed before.

## Docs and ADRs

data.source.coop: ADR-013 as revised in
source-cooperative/data.source.coop#234 is what this implements. The
route contract here (`{key_hash}` in, always-200 standing out,
proxy-as-itself auth) is the seam that
source-cooperative/data.source.coop#235 calls;
source-cooperative/data.source.coop#235 is the proxy PR replacing
source-cooperative/data.source.coop#233. The ADR-005 amendment in
source-cooperative/data.source.coop#234 records the sentinel subject.

docs.source.coop: the unattended-workflow guide
(source-cooperative/docs.source.coop#34) carries the setup the show-once
view prints, and uses the same variables. Nothing existing describes
keys.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01R1eiTse4416N6uTgAy4Ddd

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
…ch key is which

A key is shown once and stored only as a hash, so nothing on the service account's page tied a record to the key in someone's environment. The record now keeps the key's last four characters, set at issue, and the key list shows it as `sck_…Xy9Q`; the issue dialog says how it will be listed. Four characters are 24 of the key's 256 random bits. The hint confirms a key in hand against its record and is never used to look one up, since keys across the platform will share it. Keys issued before this have no hint and are listed without one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@alukach

alukach commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main and force-pushed; the branch is its four commits on main with no merge commits. New commit on top: each API key's record keeps its last four characters, and the key list shows it as sck_…Xy9Q, so a key in someone's environment can be matched to its record and service account. Keys issued before this are listed without a hint. The description is rewritten for the opaque-key design #580 brought in (it still described proxy-signed keys), and the title now says opaque.

… menu

Each key's row ran its hint and three dates together in one wrapping line, beside two action links. Now the label and its Revoked or Expired marker sit over the key's hint alone; to the right, two short lines say how it has been used and when it ends ("Used 3 days ago", "Expires in 5 months", "Revoked 9 months ago"), with the exact dates, who issued it and when, in their tooltip; and Change expiry and Revoke move into a "⋯" menu. A revoked key keeps an invisible copy of that button so its lines align with the others.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@alukach

alukach commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

New commit: the key rows were congested, with the hint and three dates wrapping in one line beside two links. Each row now shows the label and hint on the left, two short relative lines on the right ("Used 3 days ago" / "Expires in 5 months"), the exact dates in a tooltip, and Change expiry and Revoke in a "⋯" menu. Description and screenshot updated.

The key list lived inside ServiceAccountDetail, so Storybook showed it only as two rows of that page. ApiKeyList now holds the rows, their expiry dialog and revoking, and the page renders it in its API keys section. Its stories cover every state a key can be in — live, never expiring, never used, expired, revoked — plus a single key, a key from before hints were recorded, and none. Their dates are set relative to today, so the rows read the same whenever the story is opened. Like its neighbours, it uses useActionState, so the Jest smoke test skips it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@alukach

alukach commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

New commit: the API key list is its own component, ApiKeyList, with stories for every key state (live, never expiring, never used, expired, revoked), a single key, a key without a hint, and none: https://source-coop-ui-git-feat-service-account-keys-radiantearth.vercel.app/?path=/story/features-service-accounts-apikeylist--default. The account page renders it unchanged.

This branch was successfully deployed

2 active deployments
Preview – source-cooperative — eea0e0ad Deployed Sep 29, 2026 by vercel[bot]
Preview – source-coop-ui — eea0e0ad Deployed Sep 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blocked Waits on another PR merging first feat

Projects

None yet

Development

Successfully merging this pull request may close these issues.

API key lifecycle for Service Accounts Integrations model for Service Accounts

1 participant