ci: add competitive CodSpeed benchmarks vs pySBD and punkt

[yisding]

What

Adds benchmarks/test_competitive_codspeed.py — CodSpeed benchmarks comparing sentencesplit's per-call latency and a throughput proxy against pySBD (the rule-based library it derives from) and punkt (nltk's statistical tokenizer), and wires it into the existing CodSpeed workflow.

• pysbd + nltk added to the isolated bench group (both pure Python).
• A punkt-model download step runs outside CodSpeed instrumentation.
• Each engine is constructed once in a fixture, so only steady-state segmentation is timed; all three get byte-identical input.
• 12 new benchmarks: segment × {short, medium, large} × {ours, pysbd, punkt} + throughput × 3.

Why only these two peers

CodSpeed's instrumentation mode measures CPU instructions, a fair proxy for wall-clock only among libraries of the same execution character. All three here are pure Python → apples-to-apples. C/Cython tokenizers (spaCy, blingfire, stanza) would have a wildly different instruction-to-wallclock ratio and are intentionally excluded — use a wall-clock harness (the compare-segmenters skill) for those.

Reading the results

CodSpeed compares each benchmark to itself across commits, not library-to-library, so the three engines are compared by reading their absolute numbers side-by-side in the dashboard (the [size-ours] / [size-pysbd] / [size-punkt] rows). The per-PR delta still guards each engine against regression.

Honest preview (local walltime — directional, CI instruction-count will differ)

case	ours	pySBD	punkt
segment short	215 µs	69 µs	2.6 µs
segment medium	1.0 ms	336 µs	22 µs
segment large	17.3 ms	30.7 ms	2.8 ms
throughput (~10 KB)	42 ms	157 ms	7.5 ms

Takeaways, unspun:

• vs pySBD: we're slower on short/medium (higher fixed per-call cost) but faster on large text and throughput (~3.7× on the throughput doc) — we scale better per-character.
• vs punkt: punkt is dramatically faster everywhere (a single trained-model pass vs. many regex passes) — the expected cost of rule-based precision. This benchmark is latency-only; it does not capture the accuracy trade-off that motivates a rule-based engine.

Isolation / safety

• Main CI targets tests/ only — never collects these files or installs pysbd/nltk into the test matrix.
• Smoke-verified locally: all 22 benchmarks (10 latency + 12 competitive) collect and pass; ruff clean.

https://claude.ai/code/session_01RBqMKEGgvcZEAXLTPP5FiV

---

Generated by Claude Code

[codspeed]

Merging this PR will not alter performance

✅ 10 untouched benchmarks
🆕 12 new benchmarks

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
🆕	test_segment[large-ours]	N/A	71.9 ms	N/A
🆕	test_segment[large-punkt]	N/A	11.1 ms	N/A
🆕	test_segment[large-pysbd]	N/A	162.5 ms	N/A
🆕	test_segment[medium-ours]	N/A	4 ms	N/A
🆕	test_segment[medium-punkt]	N/A	594 µs	N/A
🆕	test_segment[medium-pysbd]	N/A	3.1 ms	N/A
🆕	test_segment[short-ours]	N/A	1.8 ms	N/A
🆕	test_segment[short-punkt]	N/A	276.9 µs	N/A
🆕	test_segment[short-pysbd]	N/A	1.4 ms	N/A
🆕	test_throughput[ours]	N/A	178.9 ms	N/A
🆕	test_throughput[punkt]	N/A	27.6 ms	N/A
🆕	test_throughput[pysbd]	N/A	861.4 ms	N/A

---

_{Comparing claude/codspeed-competitive (553a36e) with main (772fda5)}