feat: score ordinal rows by rank, each figure against its own null

[TMHSDigital]

Fixes #6. This follows the approved design, with the METHODOLOGY section ("Ordinal score questions are scored by rank") written first as the spec.

Measurement (src/plumbline/metrics/ordinal.py)

The answer. A score answer is a distribution over levels 0 to K−1. Its point answer is the expected score, the vendor's own answer, never swapped for the argmax. An arm that returns only a level gets that level as its expected score, and only the first figure below.

The three figures:

figure	what it measures	read against
Mean absolute error, in levels	rank-aware accuracy: 1 level off costs 1, 3 off costs 3	a permutation null: the arm's own expected scores shuffled across rows. A uniform-guess null was rejected, because always answering the middle level would beat it.
Ranked probability score	the ordinal counterpart of Brier, and a proper scoring rule	calibrated-model floor
Cumulative calibration error	ordinal calibration: at each threshold, P(level ≤ k) against how often it was, pooled and binned as ECE is	calibrated-model floor

The floors. The predicted distributions are held fixed and each row's gold level is redrawn from its own distribution, vectorized over the draws, so every resample is calibrated by construction. The wording reuses _judgment, so INCONCLUSIVE reads exactly as it does everywhere else.

Tests (tests/test_ordinal.py, 10 tests):

• The score charges by distance: exact 0, near miss 1/3, far miss 1.
• Events are counted per threshold.
• A calibrated arm exceeds each floor's 95th percentile at most 4 times in 20 seeds. That's the promise a floor makes, so one seed proves nothing either way.
• An overconfident arm clears both floors.
• An informative arm beats the permutation null, and a constant middle-level arm doesn't.
• The statements carry row counts and floors.
• An arm with no distributions gets the error only, and says why.
• End to end on the fixture through the CLI: the score block is separate, and the choice ECE is unchanged at 0.0740 over 105 rows.

Wiring

• Types. SUPPORTED_QUESTION_TYPES includes score, and a new CHOICE_QUESTION_TYPES names the rank-blind group.
• Loader.
  • A JevBench score row's rubric (a list, one entry per level) becomes the levels' descriptions, where before it was dropped without being counted.
  • A score row whose options aren't integer levels 0 to K−1 is refused.
  • The exclusion note is replaced, not deleted.
• typesafe_wire.
  • Asks a real Score with the rubric in level order and reads the vendor's expected score as given. label is the nearest level, for display only.
  • A distribution over different levels, or an expected score outside them, is a contract error; nothing is clamped.
  • score_instructions joins the cache key only when changed, so every existing key stands (tested).
• The other arms answer the levels as options and say so in asked_as.
• Report.
  • _arm splits into a choice block and a #### Score questions block, and the choice block's early return can no longer swallow the score block.
  • Cost counts score rows, since they're real calls.
  • The score block says it isn't comparable with the choice figures, and that recalibration and the cascade don't apply to it.
• RunResult accessors. probabilities(), outcomes, accuracy and the rest are "the calibratable column", so they now read choice rows only, through choice_successes. The site build caught this: its worked example started deriving ECE over 111 rows, and parity refused the deploy.

What changes for users

• The public fixture. All 111 rows now run. The report gains a score block, all INCONCLUSIVE on 6 rows (true at that size). The choice figures are unchanged over 105 rows, and the dataset hash moves from 1b96dc91 to 420956a9.
• Docs. docs/example-report.md is regenerated. README's limitation bullet and quickstart sample are updated, as are docs/datasets.md (the score row format), datasets/public/README.md, PLAN (Score and ordinal support: rank-aware metrics for rows that currently load and are excluded #6 landed) and the site's worked-example heading.

The full suite is 620 passed. mypy --strict passes, and so does the site gate (parity including the worked example, links, search, browser smoke). The prose checks pass.

🤖 Generated with Claude Code