Skip to content

ci: replace MinIO with RustFS as the local/CI S3 backend - #148

Merged
alukach merged 2 commits into
mainfrom
ci/rustfs-backend
Sep 24, 2026
Merged

alukach merged 2 commits into
mainfrom
ci/rustfs-backend

Conversation

@alukach

@alukach alukach commented Sep 24, 2026

Copy link
Copy Markdown
Member

What I'm changing

The Integration Tests job fails on every branch. It stops at the "Start MinIO" step, before any code under test runs:

docker: Error response from daemon: pull access denied for minio/minio, repository does not exist or may require 'docker login'

MinIO has stopped publishing public images. Docker Hub refuses minio/minio and minio/mc. The quay.io mirrors also refuse anonymous pulls: quay.io/minio/minio returns 401 since about 2026-09-24. dl.min.io, where CI downloads the mc binary, returns 410. The only image still public, quay.io/minio/aistor/minio, starts but denies every S3 operation without a license. The last green CI run was on 2026-07-15.

This PR replaces MinIO with RustFS, which is Apache-2.0 and at release 1.0.0. I compared RustFS, VersityGW, SeaweedFS and Garage with a probe of every S3 feature the proxy uses. RustFS passes all of them, including conditional PUT (If-Match / If-None-Match); the proxy forwards those headers to the backend since #122. RustFS also accepts the existing minioadmin static credentials, so no wrangler config changes. Garage does not enforce conditional PUTs, which rules it out.

How I did it

  • .github/workflows/ci.yml: "Start MinIO" becomes "Start RustFS". It runs rustfs/rustfs:1.0.0 with the tag pinned, so a registry change can't break every branch at once. The step polls /health. "Seed buckets" uses the aws CLI that is preinstalled on runners instead of mc.
  • docker-compose.yaml: the services are now rustfs and rustfs-init. The healthcheck curls /health. The seeder runs on amazon/aws-cli:2.37.2 and is idempotent: it skips mb for buckets that already exist, so docker compose up works on a volume that already has data.
  • Dropped mc anonymous set download: the proxy already enforces anonymous access (anonymous_access = true) and always signs backend requests with the configured keys. A public bucket policy on the backend was never needed.
  • scripts/integration-test.sh, Makefile, CONTRIBUTING.md, docs/getting-started/*: rename MinIO to RustFS in messages and docs. The readiness probe now uses /health. Mentions of MinIO as a user-configurable backend (for example docs/configuration/buckets.md) are unchanged because they describe production setups, not the local fixture.

Test plan

  • scripts/integration-test.sh locally against RustFS: 37 passed, 7 skipped. The skipped tests are OIDC tests that only run in GitHub Actions.
  • docker compose up: RustFS is healthy and the seeder creates both buckets and exits 0.
  • The same commit on feat(sts): GetCallerIdentity + real configure-aws-credentials integration test #126 passed all CI jobs, including Integration Tests (run 36070977529).
  • "Preview / Smoke Test" is still red. That is not caused by this PR: TestRangeRequests has failed on every branch since at least 2026-08-31.

🤖 Generated with Claude Code

MinIO no longer publishes public images: Docker Hub and quay.io
(minio/minio, minio/mc) now refuse anonymous pulls and dl.min.io returns
410, so the Integration Tests job failed at "Start MinIO" on every branch.

RustFS (rustfs/rustfs:1.0.0, Apache-2.0) passes the S3 features the proxy
relies on, including conditional PUT, and accepts the same minioadmin
credentials, so wrangler configs are unchanged. Buckets are seeded with the
aws CLI instead of mc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@claude

claude Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Claude finished @alukach's task in 1m 33s —— View job


Review complete

  • Gather context (diff, changed files)
  • Review for correctness/security/best-practice issues
  • Apply ponytail-review pass
  • Post findings

✅ No blocking issues — safe to merge.

Verified the key claims in the PR description against the code:

  • The backend bucket for public-data is configured with access_key_id/secret_access_key in examples/cf-workers/wrangler.integration.toml:29-33, confirming the proxy always signs backend requests — dropping mc anonymous set download doesn't remove any access path actually used by the proxy or the test suite (tests/integration/test_integration.py's direct-backend client uses minioadmin creds, not anonymous).
  • docker-compose.yaml's seed loop (s3 ls "s3://$$b" || s3 mb ...) is correctly idempotent for repeated docker compose up; the $$ escaping is required and correct for Compose's variable interpolation of the YAML.
  • ci.yml's seed step doesn't need the same idempotency check since it runs in a fresh container each time.
  • The readiness polling loops (CI and scripts/integration-test.sh) preserve the same non-failing behavior as the prior MinIO version (pre-existing weakness, not introduced by this PR).

Nothing else stood out — env var names, credentials, and health-check endpoints are consistent across ci.yml and docker-compose.yaml.

Simplify (ponytail)

Nothing to cut — the s3() shell helper in both ci.yml and docker-compose.yaml is a reasonable one-liner, not a reinvention, and the two copies live in genuinely different execution contexts (GH runner vs. init container) so sharing isn't worth it.


💰 Estimated review cost: $0.33 · 1m33s · 12 turns

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

📖 Docs preview deployed to https://multistore-docs-pr-148.development-seed.workers.dev

  • Date: 2026-09-24T23:18:24Z
  • Commit: 131b3d9

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alukach
alukach marked this pull request as ready for review September 24, 2026 23:44
@alukach
alukach merged commit 1b63f73 into main Sep 24, 2026
18 of 20 checks passed
@alukach
alukach deleted the ci/rustfs-backend branch September 24, 2026 23:44
@alukach alukach mentioned this pull request Sep 25, 2026
7 tasks done

This branch was successfully deployed

1 active deployment
preview — c945df8d Deployed Sep 24, 2026 by alukach via Deploy & Test / Deploy #414
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant