feat(abbr)!: V2 single-pass abbreviation engine + tech-debt paydown (major, breaking)

.github/workflows/python-package.yml

@@ -41,6 +41,13 @@ jobs:
       - name: Test with pytest
         run: uv run --python ${{ matrix.python-version }} pytest --cov=sentencesplit tests/ --color yes
 
+      - name: Test benchmark harness (path redaction)
+        # The corpus-compare harness lives under benchmarks/ (never shipped), so its
+        # correctness tests sit outside testpaths=["tests"]. Run them explicitly so the
+        # path-redaction guard keeps running. pytest's pythonpath=["."] puts the repo
+        # root on sys.path for `from benchmarks.corpus_compare import segmenters`.
+        run: uv run --python ${{ matrix.python-version }} pytest benchmarks/test_corpus_compare_segmenters.py --color yes
+
   free-threaded-test:
     runs-on: ubuntu-latest
 
@@ -180,9 +187,9 @@ jobs:
               if arc not in names:
                   errors.append(f"missing language module {arc}")
 
-          # (c) no test/analysis/benchmark files may leak into the wheel.
+          # (c) no test/benchmark files may leak into the wheel.
           for name in sorted(names):
-              if name.startswith(("tests/", "analysis/", "benchmarks/")):
+              if name.startswith(("tests/", "benchmarks/")):
                   errors.append(f"leaked non-shipping path {name}")
 
           if errors:
@@ -194,6 +201,6 @@ jobs:
           print(f"Wheel contents OK: {wheel}")
           print(
               f"  py.typed shipped, {len(expected)} language modules present, "
-              "no test/analysis/benchmark leakage"
+              "no test/benchmark leakage"
           )
           PY

.gitignore

@@ -108,11 +108,8 @@ ENV/
 # vscode
 .vscode/
 
-# Regenerable benchmark/analysis JSON dumps (produced by sibling
-# analysis/benchmark scripts; not consumed by tests or the package).
-analysis/*comparison*.json
-analysis/*results*.json
-analysis/verdicts.json
+# Regenerable benchmark JSON dumps (produced by the comparison harness;
+# not consumed by tests or the package).
 benchmarks/corpus_compare/results/divergences_all.json
 benchmarks/corpus_compare/results/verdicts.json
 

AGENTS.md

@@ -18,8 +18,7 @@ Tests are in `tests/`:
 Docs and research assets:
 - `README.md`: install, public API, lookahead behavior, multi-language usage, and spaCy integration.
 - `CONTRIBUTING.md`: contribution workflow and TDD guidance.
-- `analysis/`: comparison scripts plus checked-in JSON/Markdown reports.
-- `benchmarks/`: benchmark helpers and golden-rule benchmark data.
+- `benchmarks/`: benchmark helpers, the cross-library comparison harness, and golden-rule benchmark data.
 - `examples/`: runnable examples, including the spaCy component and timing script.
 
 ## Build, Test, and Development Commands

CLAUDE.md

@@ -60,6 +60,6 @@ CI runs lint + tests on Python 3.11, 3.12, 3.13, 3.14.
 - Ruff is the sole linter/formatter. Line length: 127.
 - `snake_case` functions/variables, `PascalCase` classes, `UPPER_SNAKE_CASE` constants.
 - Commit messages must follow [Conventional Commits](https://www.conventionalcommits.org/) — `python-semantic-release` parses them to group changelog entries. Format: `<type>(<optional scope>): <imperative subject>`. Common types: `feat`, `fix`, `perf`, `refactor`, `docs`, `test`, `build`, `ci`, `chore`. Breaking changes use `!` after the type/scope (e.g. `feat!: drop Python 3.10`) or a `BREAKING CHANGE:` footer. Keep the subject short, imperative, and specific.
-- The release workflow (`.github/workflows/release.yml`) does **not** derive the version number from commit types: the version bump is chosen manually from a `workflow_dispatch` dropdown (`patch` / `minor` / `major` / `prerelease`), which forces the corresponding level. Conventional commit types only drive changelog grouping. Therefore the person cutting a release must pick the right level by hand: a cycle containing any `feat:` must be released as a **minor**, and any breaking change as a **major**. (The next release must be **0.1.0** (minor) — `feat: support free-threaded Python` landed since v0.0.5.)
+- The release workflow (`.github/workflows/release.yml`) does **not** derive the version number from commit types: the version bump is chosen manually from a `workflow_dispatch` dropdown (`patch` / `minor` / `major` / `prerelease`), which forces the corresponding level. Conventional commit types only drive changelog grouping. Therefore the person cutting a release must pick the right level by hand: a cycle containing any `feat:` must be released as a **minor**, and any breaking change as a **major**.
 - Bug fixes get a regression test in `tests/regression/` before the fix.
 - Public API changes (lookahead, split_mode, spans) go in `tests/test_segmenter.py`.

README.md

@@ -80,7 +80,7 @@ seg.segment_spans("My name is Jonas E. Smith. Please turn to p. 55.")
 #  TextSpan(sent='Please turn to p. 55.', start=27, end=48)]
 ```
 
-`segment_spans()` always returns `TextSpan` objects with `.sent`, `.start`, `.end` regardless of the `char_span` constructor flag.
+`segment_spans()` always returns `TextSpan` objects with `.sent`, `.start`, `.end`; `segment()` always returns plain strings. Spans are byte-for-byte faithful: every span is an exact slice of the source and reassembling them reproduces it verbatim.
 
 ### Streaming / lookahead
 
@@ -126,7 +126,7 @@ stream.feed(full_text)
 assert stream.get_completed_sentences() + stream.flush() == Segmenter(language="en").segment(full_text)
 ```
 
-`StreamSegmenter` accepts the same `language` / `clean` / `char_span` / `split_mode` params as `Segmenter`, plus a streaming-specific `buffering_mode` (`"conservative"` (default) / `"balanced"` / `"aggressive"`) and an optional `max_buffer_size` guard against an unbounded tail.
+`StreamSegmenter` accepts the same `language` / `clean` / `split_mode` params as `Segmenter`, plus a `char_span` flag selecting `TextSpan` vs plain-string output, a streaming-specific `buffering_mode` (`"conservative"` (default) / `"balanced"` / `"aggressive"`), and an optional `max_buffer_size` guard against an unbounded tail.
 
 See [examples/streaming_to_tts_recipe.py](examples/streaming_to_tts_recipe.py) for a runnable LLM-to-TTS recipe.
 
@@ -251,7 +251,7 @@ seg = sentencesplit.Segmenter(language="en", clean=False)
 seg.segment("My name is Jonas E. Smith. Please turn to p. 55.")
 ```
 
-`Segmenter(language=..., clean=..., char_span=...)`, `segment()`, and the `TextSpan` fields (`.sent`, `.start`, `.end`) all behave as they do in pySBD, and the English [Golden Rules](https://github.com/diasks2/pragmatic_segmenter#the-golden-rules) pass identically. What you gain on top:
+`Segmenter(language=..., clean=...)`, `segment()`, and the `TextSpan` fields (`.sent`, `.start`, `.end`) all behave as they do in pySBD, and the English [Golden Rules](https://github.com/diasks2/pragmatic_segmenter#the-golden-rules) pass identically. The one break: pySBD's `char_span=True` constructor flag is gone — call `segment_spans()` for `TextSpan` output instead (`Segmenter(char_span=True).segment(text)` → `Segmenter().segment_spans(text)`). What you gain on top:
 
 - **Streaming/lookahead** — `segment_with_lookahead()` / `should_wait_for_more()` for incremental input, plus the higher-level [`StreamSegmenter`](#streaming-segmentation) feed/flush wrapper for token-by-token sources (LLM output, ASR partials).
 - **`split_mode`** — a `"conservative"` / `"balanced"` / `"aggressive"` bias for ambiguous boundaries (`"balanced"` is the default and matches the historically tuned output).
@@ -375,6 +375,16 @@ from `sentencesplit.languages` and the documented ISO 639-1 language codes (see
 `Processor` internals, and the nested language hooks — is private and may change
 without notice.
 
+**spaCy component.** The package registers a `spacy_factories` entry point so spaCy
+users can do `nlp.add_pipe("sentencesplit")` (see [spaCy integration](#spacy-integration)).
+The *stable* contract is the registered factory name `"sentencesplit"` and its
+`language` config option, both of which follow the SemVer policy above. The underlying
+Python factory (`sentencesplit.spacy_component.create_sentencesplit` and the
+`SentenceSplitFactory` class) is deliberately **not** in `sentencesplit.__all__`: its
+call signature tracks spaCy's factory protocol rather than this library's API, so it may
+change with spaCy's requirements without a SemVer bump here. Add the component by name —
+do not import or subclass the factory directly.
+
 **Output stability.** Sentence segmentation output is *not* part of the frozen API.
 It MAY change in minor or patch releases when the change is a net accuracy
 improvement; any such output change is recorded in [CHANGELOG.md](CHANGELOG.md).