Skip to content

feat(docs): the promise on line 41 is derived from a matrix that asks the doorstep, and the quick start is four commands a reader can run (#592, #593) - #595

Merged
stephrobert merged 1 commit into
mainfrom
docs/592-593-capability-matrix-and-quickstart
Aug 28, 2026
Merged

feat(docs): the promise on line 41 is derived from a matrix that asks the doorstep, and the quick start is four commands a reader can run (#592, #593)#595
stephrobert merged 1 commit into
mainfrom
docs/592-593-capability-matrix-and-quickstart

Conversation

@stephrobert

Copy link
Copy Markdown
Owner

Closes #592. Closes #593.

The contradiction, read in the files before anything moved

README.md:41           Run your Terraform against Scaleway, Outscale or Exoscale
docs/confidence.md:48  A Terraform run against Exoscale | no, and refused rather
                       than half-served
exoscale/veto.go       returns a non-empty reason for terraform and opentofu
internal/cli/up.go:384 consults it before a process starts

Hand-written prose, outside every marker — which is exactly why no gate could
see it. feint docs --check verifies numbers and names, each comparable to an
artefact. "Run your Terraform against …" is a claim about capability, owned
by nobody, and it had been false since #525 landed on 2026-08-26.

The sentence, and why it stays a promise rather than a warning

**Point Terraform and the official cloud CLIs at your own machine.**
No cloud account, no credentials, and nothing created anywhere.

Terraform and OpenTofu drive Scaleway and Outscale. Each pack also has its
own official CLI, and every one of them drives this emulator end to end:

- **Scaleway** with `scw`.
- **Outscale** with `octl`.
- **Exoscale** with `exo`. Terraform and OpenTofu join it the day a published
  release carries the fix for exoscale/terraform-provider-exoscale#573, which
  `feint up` refuses at the doorstep until one does.

The headline is unqualified and leads; the Exoscale line opens with what
works
and the engine is a future addition, not a missing feature; the
exception is one clause of one bullet. And it says "the day a published release
carries the fix"
#573 is closed and PR #576 is merged into master, but no
tag carries it (last is v0.70.0, 17 July).

The list shape is not decoration. The claim reader treats a unit naming two
packs and two clients as asserting every pair, so a single sentence reading
"scw for Scaleway, octl for Outscale, exo for Exoscale" would be claiming
scw against Exoscale. One pack per item is what makes each line carriable.

The matrix, and what proof demands

internal/cli/capability.go — provider × client × mode × support × proof ×
reason, published as a generated table in docs/clients.md.

proof is resolved, never printed: conformance workflow resolves through
the same scan the status table already shares; up.go VetoEngine resolves
through the pack itself, via newServer + packEngineVeto — the code up and
down consult.

Checked in both directions: a row nothing establishes fails, and a pair an
instrument establishes with no row fails too.
A supported row for an engine a
pack really vetoes is refused by name. A refused row must carry a reason and a
marker, and the reason must contain the marker — a rule found when the claim
reader fired on the matrix's own published table.

docs:check red on the restored sentence, then green again:

$ put_back_the_lie.py README.md && ./feint docs --check
exit=2
README.md, in the generated `promise` block, puts Terraform and Exoscale together
  and capabilityMatrix refuses that pair: say why by naming
  exoscale/terraform-provider-exoscale#573, or stop claiming it

An ordering defect in feint docs itself, found on the way

It wrote its target last, from a copy spliced before the helpers that re-read
the same file — so a run changing two README sections kept one and printed
success. TestARunThatChangesTwoSectionsOfTheREADMEKeepsBoth; reverting the
ordering makes it fail.

The quickstart, and its gate

examples/quickstart/{scaleway,outscale}34 and 43 HCL lines,
runtime: mode: off. Outscale is three over budget: the provider block plus the
create-then-link public IP its API requires. The Net+Subnet form was measured
first (52 lines) and dropped after proving a Vm applies and re-plans empty
without one. examples/stacks/ is unchanged — it stays the qualification
stack that found #249 and #250.

tools/conformance/quickstart.sh + conformance:quickstart, wired into the
aggregate and onto the terraform/opentofu legs. It runs feint up, checks every
declared ready condition was confirmed, checks the emulator answers, checks the
second plan is empty, then feint down and that nothing answers.

Real execution:

scaleway: feint up → ok ×4 | second plan empty | feint down → ok ×2
outscale: feint up → ok ×3 | second plan empty | feint down → ok ×2

A defect the first real run found: the script guessed its engine (tofu if
installed) while feint up runs the declared one. On a station holding both,
it planned with OpenTofu a directory Terraform had initialised — Inconsistent
dependency lock file
, reported as "the second plan errored". It reads
iac.engine from the declaration now, the way up does.

And #593's own complaint is dead: the Apply complete! line is derived from
the configuration (resourceCount, refusing count/for_each rather than
miscounting), and the gate lifts that line out of README.md and requires the
run to print it.

No proof.json, and the header says why: a quickstart that boots nothing can
honour none of #503's families.

The French block

The generated blocks carrying prose are now rendered per locale.
TestTheFrenchQuickStartIsInFrench names the five English sentences that were
landing there and asserts the French ones are present and the commands
untranslated. The > [!NOTE] on README.fr.md was itself false and is
corrected.

Falsification

capability-matrix.json13 mutations, all bite. Two survived the first
replay, and both times the comment was wrong: one asserted merely that
"something was reported" where a second correct rule fired about the same table
(it now asserts what the refusal says); the other named the wrong branch. Both
rewritten rather than defended.

stack-proof.json — 3 mutations killed by the refactor, retargeted, all 5 bite.

Outside the tree, on a copied layout: the quickstart gate reddens on an
unreachable ready condition, on a non-empty second plan, on an example declaring
no ready condition, and on a README claiming a resource count the run does not
produce. Control green each time.

Gates

prepush · docs:check (and red on the restored sentence) · lint-shell ·
go test ./... · conformance:quickstart · conformance:environment ·
conformance:leg -- probe · conformance:leg -- fields (352/375, 216 s) · the 7
specs testplan named · falsify:lint (944 mutations, 148 specs) — all green.

internal/providers/ untouched; every run on 127.0.0.1:4699; FEINT_VM never
left off.

What was not obtained

Where testplan was wrong, and it is fixed here

It sent examples/quickstart/** to conformance:environment through the
catch-all examples/ rule, and named conformance:quickstart nowhere
while testplan:check passed, because a wrong rule is still a rule. Two rules
added, each with a cited claim and an Unproven line saying the suite judges what
a reader copies and never a machine that boots.

… the doorstep, and the quick start is four commands a reader can run (#592, #593)

Measured on `main@3b00d23`, and verified in the files before touching them:
`README.md:41` promised "Run your Terraform against Scaleway, Outscale or
Exoscale" while `docs/confidence.md:48` answered "A Terraform run against
Exoscale — no, and refused rather than half-served", and the Exoscale pack's
VetoEngine had been refusing `iac.engine: terraform` since #525 landed on
2026-08-26. Every doc gate was green throughout, and none was wrong: they
compare values an artefact also holds, and a claim about capability is not one.

capabilityMatrix is provider × client × mode × support × proof × reason. The
proof column is resolved, never printed: a `supported` row against
clientsProvenInCI — the same workflow scan the status table reads — and a
`refused` row against the pack's own VetoEngine, which is the code `up` and
`down` consult. Both directions, so a veto nobody wrote down and a row nothing
proves are equally refused. The promise is generated from it per locale, and
unownedCapabilityClaims reads every generated block of the front pages back: a
sentence naming a refused pair passes only if it carries the upstream issue
that would change it. Put the old sentence back and `docs --check` exits 2,
naming the pair and the fix.

The Quick Start taught the 0.10 sequence and was not copyable — no directory,
no main.tf, no provider block, under an `Apply complete! Resources: 5 added`
unreachable from it. examples/quickstart/{scaleway,outscale} are the first
example now: 34 and 43 lines, `runtime: mode: off`. examples/stacks is
unchanged and keeps being the qualification stack. The apply line is counted
from the configuration, and tools/conformance/quickstart.sh lifts it out of the
README and requires the run to print it. The quickstart directory joins the
population `feint docs --check` already judges, rather than living one
directory away from it.

Run: prepush green, docs:check green and red on the restored sentence,
conformance:quickstart, conformance:environment, conformance:leg -- probe and
-- fields on 127.0.0.1:4699, lint-shell, and the falsification specs testplan
named for this diff. tools/falsify/specs/capability-matrix.json is 13
mutations and every one bites; three more, planted outside the tree, redden the
quickstart gate. limits:check is red on #570 and was before this branch.

Assisted-by: Claude Code (claude-opus-5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant