[factory] Stand up `factory-test-infra`: Cloudflare-based ephemeral verification environments (Workers for Platforms + Containers) + factory provider wiring

[khaliqgant]

Part of Epic: Factory Verification & Environments (#141). This is the tier-5/6 data-plane home. The provider (#142) and gate (#145) run on what this stands up. Best done first in Track C.

Decision: Cloudflare-first, in a dedicated factory-test-infra repo

factory is a library (@agent-relay/factory) — it must not own or operate the environment substrate. The org is already all-in on Cloudflare + SST + wrangler + workerd/miniflare, so tier-5/6 verification environments run on Cloudflare, not a bespoke Kubernetes cluster:

• Ephemeral per-feature environment → a Workers for Platforms dispatch namespace (isolated worker deployments, created and torn down as a unit — the role a K8s namespace would have played).
• Non-Worker services (Postgres, Redis, a non-JS API) → Cloudflare Containers, with Hyperdrive / D1 / KV / R2 / Queues for state.
• CI tier → @cloudflare/vitest-pool-workers / miniflare (cloud already runs this — test:integration, miniflare-sage).
• Load → k6 run from a Container/Worker against the environment URL (see Load-simulation harness: in-cluster load generation with SLO gates and evidence #144).

Kept in its own repo (not merged into cloud's infra/) so the spiky, ephemeral verification account/config is isolated from product infra for blast-radius + cost accounting. K8s/Daytona remain optional escape-hatch providers behind the same EnvironmentProvider port (#142) for the minority of target stacks that genuinely can't run on Cloudflare — Cloudflare Containers is not a general Kubernetes replacement for arbitrary heavy container stacks.

⚠️ Repo creation is an owner action — factory-test-infra does not exist yet (the GitHub App is denied org repo creation: 403 Resource not accessible by integration). First step: an owner creates agentworkforce/factory-test-infra (private), or grants the app repo-creation. Everything below is its intended contents.

What (in factory-test-infra)

1. Cloudflare account/config as code — SST (or wrangler + Terraform Cloudflare provider) for: the Workers for Platforms dispatch namespace(s) used for verification, Containers definitions for the non-Worker services, and the D1/KV/R2/Queues/Hyperdrive bindings a verification environment can request.
2. Isolation — one dispatch namespace per environment; per-environment bindings/secrets scoped so environments can't read each other or prod. (Dispatch-namespace isolation replaces NetworkPolicies.)
3. Quotas & cost caps — per-environment resource limits (Container instance sizes/counts, request caps), a max concurrent environments cap, and a per-run cost/time budget that aborts + reaps runaway runs. Use Cloudflare usage/analytics + account budgets.
4. Reaper ops — a scheduled sweep (Cron Trigger) that deletes dispatch namespaces / Container instances by ownership/age labels (belt-and-braces with the provider TTL reaper in [factory] EnvironmentProvider port + Cloudflare ephemeral-environment provider (Workers for Platforms + Containers); K8s/Daytona optional #142); leak alerting.
5. Base templates — wrangler/Container scaffolding that target repos' verification-stack descriptors (Verification-stack descriptor contract + deployer: repos declare their full stack, Factory brings it up #143) render into.

What (in factory)

6. Provider wiring — CloudflareEnvironmentProvider ([factory] EnvironmentProvider port + Cloudflare ephemeral-environment provider (Workers for Platforms + Containers); K8s/Daytona optional #142) reads the factory-test-infra account/API-token config (via a Resource.*-style secret, not process.env); a reusable CI workflow that runs the Track-C E2E suites ([factory] EnvironmentProvider port + Cloudflare ephemeral-environment provider (Workers for Platforms + Containers); K8s/Daytona optional #142–Verification gate: provision → deploy → E2E → load → evaluate → teardown, wired into the merge gate #145) against miniflare + a real dispatch namespace, and tears down.

Acceptance

• factory-test-infra repo exists with Cloudflare account/config as code (dispatch namespace(s), Containers, bindings) documented + scripted; miniflare path for CI.
• One dispatch namespace per environment; per-env bindings/secrets isolate environments from each other and prod.
• Per-env limits + max-concurrency + cost/time budget enforced; runaway runs aborted and reaped.
• Scheduled reaper (Cron Trigger) deletes orphaned namespaces/Container instances; leak alerting.
• factory's CloudflareEnvironmentProvider connects; reusable CI workflow runs [factory] EnvironmentProvider port + Cloudflare ephemeral-environment provider (Workers for Platforms + Containers); K8s/Daytona optional #142–Verification gate: provision → deploy → E2E → load → evaluate → teardown, wired into the merge gate #145 suites and tears down.

✅ End-to-end verification — Factory success criteria (REQUIRED)

"Compiles" is not done. Provision a real dispatch namespace + Container, prove the guardrails bind, and prove CI runs the suite end to end.

Run it

# in factory-test-infra
bash ci/verification-env/up.sh        # creates a dispatch namespace + bindings + a Container service
bash ci/verification-env/run-suite.sh # runs factory #142–#145 E2E suites against it
bash ci/verification-env/down.sh      # deletes the namespace + Container

The E2E must prove the guardrails bind (not just exist):

1. Isolation: a worker/Container in env-A cannot read env-B's bindings/secrets or reach its private service (assert the failure).
2. Quota/cost cap: requesting Container instances beyond the per-env limit is refused; a run past its time/cost budget is aborted and its namespace reaped.
3. Concurrency cap: provisioning beyond max-concurrent environments is refused.
4. Reaper: an orphaned dispatch namespace (owner label, past age) is deleted by the Cron sweep.
5. Load scale: a k6 load run (Load-simulation harness: in-cluster load generation with SLO gates and evidence #144) against a deployed Worker/Container completes and the environment absorbs it (Workers autoscale; Container instance scaling within the cap).
6. CI: the workflow runs factory [factory] EnvironmentProvider port + Cloudflare ephemeral-environment provider (Workers for Platforms + Containers); K8s/Daytona optional #142–Verification gate: provision → deploy → E2E → load → evaluate → teardown, wired into the merge gate #145 suites green against a real dispatch namespace and tears down.

Deliverable

• factory-test-infra ci/verification-env/{up,run-suite,down}.sh + reusable CI workflow, one-command.
• Guardrail assertions (isolation, quota/cost refuse, concurrency refuse, reaper delete, load absorb) each fail loudly if the guardrail is removed.

Anchor points

• New repo factory-test-infra: infra/ (SST or wrangler + Terraform Cloudflare provider), containers/ (Container service defs), ci/verification-env/ (up/run/down + miniflare), Cron-Trigger reaper worker.
• In factory: src/environments/cloudflare-provider.ts (from [factory] EnvironmentProvider port + Cloudflare ephemeral-environment provider (Workers for Platforms + Containers); K8s/Daytona optional #142) reads the account/token; src/config/schema.ts env config; .github/workflows/ reusable verification workflow.
• src/orchestrator/reaper.ts (identity-checked reaper pattern to mirror).
• Precedent for the Cloudflare/SST/miniflare posture: relaycast-cloud (workerd + @cloudflare/vitest-pool-workers), cloud's miniflare-sage service + test:integration.

[khaliqgant]

Factory human input request

Agent: ar-146-impl-factory
Issue: 146
Question: Can an organization owner create the private AgentWorkforce/factory-test-infra repository and provide its linked Cloudflare account resource secret so the required real dispatch-namespace, Container, guardrail, load, reaper, and teardown E2E can run?

[agent-relay-code]

Factory human input request

Agent: ar-146-babysit-factory
Issue: 146
Question: Can an organization owner create the protected factory-verification environment in AgentWorkforce/factory-test-infra, add the cloudflare_resource_json Resource secret, and run the real Cloudflare job for companion PR #2 against Factory commit 52bf6b4 so the required dispatch namespace, Container, guardrail, load, reaper, and teardown evidence is produced?