[Sandbox] Sandbox-scoped credentials — no production secrets in a sandbox

[Sam123ben]

Part of the Agent Sandbox & Evaluation epic (#192). Splits the credential half out of
#180 because it is the single highest-risk part of the sandbox.

Goal

No production secret ever reaches a sandbox. An agent under evaluation is, by
definition, running unvalidated instructions against tools — treat its credentials as
if the agent were hostile.

Today (grounded)

PrincipalLLMResolver / PrincipalSCMResolver resolve the triggering user's own
connectors, and job_runner.py forwards the matching provider key into the Job env.
That is exactly right for production and exactly wrong for a sandbox: an eval run would
execute with the user's real GitHub PAT and real LLM key against real orgs.

Tasks

• SandboxCredentialResolver — resolves the sandbox's own scoped credentials, never
  the principal's production connectors, unless the sandbox spec explicitly opts a
  named connector in and the user confirms.
• LLM keys: allowed (an eval must call a model), but metered against the sandbox
  budget and attributed to the sandbox, not the user's production spend.
• SCM / cloud / GitOps credentials: denied by default. A sandbox that needs SCM
  gets a scoped test-org token, not the user's PAT.
• Sandbox ServiceAccount is distinct from the pipeline SA, with no WIF binding to
  production GCP service accounts.
• Assert it in tests: a sandbox Job spec never contains a key that came from a
  production connector.
• Audit every credential grant into a sandbox.

Acceptance

Given a user with real GitHub and Anthropic connectors, creating a sandbox and invoking
it produces a Job whose env contains the sandbox-scoped LLM key and no SCM/cloud
credential, and the grant is audited.

Plan: docs/plans/agent-sandbox-evals/IMPLEMENTATION-PLAN.md §7

[Sam123ben]

Deployment update — 2026-08-19

The application isolation slice shipped in PR #248 and is deployed from merge commit 12912ca.

Verified:

• Sandbox invocation requires an explicitly selected and confirmed personal LLM connector.
• User and tenant qualified connector ownership is enforced; team, global, shared, disabled, keyless Vertex, other-user, and cross-tenant connectors are rejected.
• Sandbox AI calls force gateway-required mode and fail closed without the configured gateway route.
• SCM, cloud/platform secrets, settings, memory/state/event buses, and production extras are stripped from sandbox dependencies.
• Sandbox token/cost metering carries sandbox, run, user, and tenant attribution with per-run limits.
• Grant audits contain identifiers only; provider errors are redacted.
• Production is on main-12912ca. Kargo promotion succeeded all 8 steps; all DevAI apps are Synced/Healthy, one ready replica each, zero restarts.
• Production DEVAI_LLM_GATEWAY_REQUIRED is true. AgentGateway, AI gateway, registry, kagent, and sandbox storage are reachable through the authenticated status path.
• The devai-sandbox ServiceAccount is distinct and has no GCP Workload Identity annotation. Sandbox Jobs disable service-account token automount.
• Local verification: 1,416 backend tests passed with 1 skipped; 274 sandbox/integration tests passed; dashboard tests, typecheck, and production build passed; Ruff and scoped strict mypy passed; all merged-commit CI/image/Trivy checks passed.

Remaining acceptance:

• Run the issue scenario end to end with a real signed-in user, a confirmed personal Anthropic connector, and a sandbox invocation, then inspect the resulting Job and audit record without exposing credential material.

Leaving this issue open until that live user acceptance is performed.

[Sam123ben]

2026-08-20 recheck: the deployed isolation implementation remains intact, but the final signed-in owner acceptance cannot be executed because no browser session is connected. No connector secret was inspected, no shared/platform key was substituted, and no identity was impersonated.

[Sam123ben]

2026-08-20 owner-session acceptance recheck: the connected in-app browser runtime returned zero available sessions (agent.browsers.list() => []). The real personal connector-owner flow cannot be truthfully executed. No shared/platform credential was substituted and no owner was impersonated; this issue remains open pending a signed-in owner session for end-to-end credential isolation acceptance.