test: reorganize suite into role-based dirs and prune redundant cases

[yisding]

Stacked on #78 (base: feat/v2-abbreviation-engine). Reorganizes the test suite by the role each test plays and acts on a multi-agent–adjudicated audit of droppable cases. Every decision that removed coverage was adversarially verified (default-to-keep) before acting — 5 initially-flagged candidates were kept after the skeptic confirmed they guard real, non-duplicated behavior.

Reorganization

tests/ root now holds only conftest.py / helpers.py (fixtures + tests.helpers imports keep working for all subdirs via the root conftest).

dir	role	count
tests/contract/	public-API, property, cross-cutting invariants	10
tests/unit/	per-module unit tests	6
tests/meta/	packaging / registry / import-hygiene guards (4 reclassified out of regression/)	8
tests/regression/	true bug-guards + snapshot/gate infra only	22
tests/data/	corpus data (abbreviation_corpus_en.py colocated)	—

The snapshot extractor (tests/regression/segment_snapshot.py) AST-scans tests/lang/ by path — lang/ is deliberately left in place so the baseline needs no path-constant change.

Pruning / fixes

• Drop tests/test_punctuation_replacer.py — asserted only internal sentinels (∯, &…&); every delimiter path is covered behaviorally by segment()-level tests.
• Rewrite the processor phase-list tests from exact __name__-tuple pins → unordered membership + a CJK-phase wiring guard + a behavioral method test.
• Dedup: challenging case 119i, the cross-file "Eq. 5" row, four byte-identical Armenian rows, Italian/Spanish duplicates.
• Drop the brittle Kazakh "unprotected" characterization (covered by regression/test_abbr_dot_normalization.py).
• Trim the dead-flag German test and correct its (inaccurate) docstring.
• Trim the docstring-substring asserts in test_processor_robustness.py.
• Reduce test_danish.py to its Danish-divergent rows; regenerate the segment snapshot (da-only removals, 140 deletions / 0 insertions — no output changes).

Move

tests/test_corpus_compare_segmenters.py → benchmarks/ (it tests the never-shipped corpus-compare harness) and wired into CI explicitly, since it now sits outside testpaths=["tests"].

⚠️ One finding worth review

The plan assumed the CJK abbreviation phase could get a segment()-level plant-a-regression test. Empirically it can't: the base initials logic subsumes it — forcing _apply_cjk_abbreviation_rules to identity leaves segment() output unchanged for every probed input. It's guarded here at the pipeline level + a behavioral unit test instead. That phase may be dead/redundant defensive code — worth a separate look.

Verification

• uv run pytest tests/ → 10502 passed, 14 skipped, 113 xfailed
• uv run pytest benchmarks/test_corpus_compare_segmenters.py → 3 passed
• ruff check / ruff format --check / mypy → clean
• python -m tests.regression.segment_snapshot --diff → no diffs

🤖 Generated with Claude Code