ESMFold2: fetch MSAs from a ColabFold server (esm ships no client — we must)

[wilke]

Ask

Let ESMFold2 obtain an MSA from a server when the user hasn't uploaded one, so
msa_mode: server works for it the way it does for Boltz / OpenFold / Chai.

Split out of #95, which covers the uploaded-A3M path.

Why it needs us rather than the tool

Boltz, OpenFold and Chai each ship their own MSA-server client, so we just pass
--use-msa-server and they fetch. The esm package has no such client — no
ColabFold, no MMseqs, no HTTP fetching anywhere in esm/utils/msa/ or
esm/models/esmfold2/. It consumes an MSA object you hand it.

So for ESMFold2 we have to do the fetching and pass the result as an A3M.
Until that exists, ESMFold2 stays excluded from the --use-msa-server
passthrough (#94) — passing a flag its CLI does not define makes click exit 2
and kills the job.

We already have most of the client

scripts/_colabfold_api_msa.py does exactly this today: submit to
{API_BASE}/ticket/msa, poll {API_BASE}/ticket/{id}, download and untar from
{API_BASE}/result/download/{id}, with a 10s poll interval and a 30-minute cap.
It was written as a fallback for when colabfold_search isn't installed.

The work is largely promotion, not invention:

1. Move it from scripts/ into the package (e.g.
   predict_structure/msa_server.py) so adapters and runners can import it.
   It is currently a standalone script with a main().
2. Wire it into the ESMFold2 path: when use_msa_server is set and no
   --msa was supplied, fetch per protein chain, write A3M, feed it through the
   ESMFold2: wire up MSA input — the model supports it and we discard it #95 upload path.
3. Add --use-msa-server / --msa-server-url to the esmfold2 subcommand, and
   remove esmfold2 from the exclusion list in App-PredictStructure.pl. The
   guard test added in Fix #75: stop passing --use-msa-server to ESMFold2, which has no such option #94 keys off whether the subcommand exposes the flag, so
   it will follow automatically.

Open question — which server?

_colabfold_api_msa.py hardcodes the public https://api.colabfold.com.
Today's matrix run confirms the server path works end-to-end from the compute
nodes (B02, O02, C02 all passed), but nothing in this repo records a
self-hosted ColabFold/MMseqs endpoint — only placeholders
(mmseqs.example.com, my-mmseqs-server.com) — and nothing is listening on
coconut. Confirm whether we are meant to use our own server, and its
address, before wiring this. If we have one, msa_server_url should default
to it rather than the public API, for all tools and not just ESMFold2.

Relevant: the public API is rate-limited and shared; a batch matrix run against
it is not polite and not reliable.

Multimer pairing

For complexes, ESMFold2 pairs rows across chains by a key= tag in the A3M
header. ColabFold emits OX=<organism_id>; upstream's tutorial says to rewrite
OX= to key= so rows sharing an organism get paired. Single-chain jobs do not
need this. Worth handling here since server-fetched MSAs are exactly where
multimer pairing matters.

Acceptance

• predict-structure esmfold2 --protein x.fasta --use-msa-server fetches and
  folds with an MSA, no uploaded file
• A matrix case with tool: esmfold2 and no msa_file exercises it
• Multi-chain jobs get per-chain MSAs, paired
• Failure to reach the server degrades to a clear error, not a silent
  single-sequence fold — the ESMFold2: wire up MSA input — the model supports it and we discard it #95 failure mode

Related

• ESMFold2: wire up MSA input — the model supports it and we discard it #95 — uploaded A3M path (prerequisite; this reuses its plumbing)
• Fix #75: stop passing --use-msa-server to ESMFold2, which has no such option #94 — the exclusion that keeps ESMFold2 jobs alive until this lands

[wilke]

Survey: how the other tools do it — don't invent a new mechanism

All three existing MSA tools delegate: the tool ships its own MMseqs2/ColabFold client and we pass flags through.

Adapter	Mechanism
boltz.py:94-97	--use_msa_server, --msa_server_url <url>
chai.py:185-188	--use-msa-server, --msa-server-url <url>
openfold.py:107-126	--use-msa-server <bool>; URL has no flag, so it writes an MSA computation settings YAML (msa_file_format, server_user_agent, server_url) and passes --msa-computation-settings-yaml

OpenFold is the useful precedent: its mechanism is unlike the others, but the interface stays identical — callers still set use_msa_server / msa_server_url and the adapter absorbs the difference. That is the shape to copy for ESMFold2.

So the work is reuse, not invention

ESMFold2 is the only tool that cannot delegate, since esm ships no client. But we do not need to write one:

scripts/_colabfold_api_msa.py already implements the ColabFold REST protocol — POST {API_BASE}/ticket/msa, poll {API_BASE}/ticket/{id}, download and untar {API_BASE}/result/download/{id}, 10s interval, 30-minute cap. It exists because colabfold_search isn't always installed locally. It is a standalone script with a main(); the change is to promote it into the package (e.g. predict_structure/msa_server.py) and call it from the ESMFold2 adapter.

Revised plan:

1. Promote, don't rewrite scripts/_colabfold_api_msa.py into the package. Keep the script as a thin wrapper so existing callers don't break.
2. Keep the existing interface. Add --use-msa-server / --msa-server-url to the esmfold2 subcommand with the same names and semantics the other three use. The adapter absorbs the difference (fetch ourselves, then feed the ESMFold2: wire up MSA input — the model supports it and we discard it #95 upload path), exactly as OpenFold absorbs its YAML quirk.
3. msa_server_url should default to the same place for every tool. Right now Boltz/Chai fall through to each tool's own default (public ColabFold) and _colabfold_api_msa.py hardcodes https://api.colabfold.com. If we run our own server, that default belongs in one place — see the open question above.

This keeps ESMFold2 indistinguishable from the other tools at the CLI and app-spec level, with the extra work confined to one adapter.

[wilke]

Deferred this week

Parking this until the ColabFold server question is answered. Not blocked on code — the plan is settled (promote scripts/_colabfold_api_msa.py into the package rather than write a fetcher, and keep the --use-msa-server / --msa-server-url interface the other three adapters already use).

The open question is operational: which server do we point at? The existing client hardcodes the public https://api.colabfold.com. It is reachable from coconut and today's matrix run passed against it (B02/O02/C02), but nothing in this repo records a self-hosted endpoint — only placeholders — and nothing is listening on coconut. A batch matrix run against the public API is neither polite nor reliable, so if we have our own, that address should become the default for all four tools, not just ESMFold2.

Until then ESMFold2 stays excluded from the --use-msa-server passthrough (#94), which is correct and safe: uploaded MSAs work via #95/#97, and server mode simply is not offered rather than failing.