perf: ~22% faster — abbreviation period pre-filter, DFA automaton, phase guards (+ investigation) - #73
Merged
Conversation
Add benchmarks/phase_profile.py (a reusable per-phase wall-time attribution harness) and analysis/SHORT_STRING_LATENCY_PLAN.md, which uses it plus a pySBD structural comparison to localize the ~22% short/medium gap vs pySBD. Findings: two fixed-cost phases dominate every call — abbreviation replacement (~40%, mostly the pure-Python Aho-Corasick scan) and list-item detection (~18%) — versus pySBD's C-level 'in' short-circuits, lock-free reads, and fewer always-on passes. The plan lays out a prioritized, behavior-preserving (byte-identical) optimization sequence (P1 abbreviation scan fast-path, P2 gate always-on regexes, P3 drop the per-call lock, P4 list-item guards) with guardrails.
Merging this PR will improve performance by 21.7%
Performance Changes
Tip Curious why this is faster? Comment Comparing 
|
Measurement (benchmarks/abbr_scan_compare.py) disproves the earlier assumption that our Aho-Corasick scan is the short-string bottleneck: input AC in winner tiny (15c) 2.4us 8.2us AC 3.4x short (87c) 12.2us 13.5us AC ~tied medium(198) 30.5us 21.0us in 1.4x large (4k) 619us 286us in 2.2x Our pure-Python AC is faster than a pySBD-style loop on short input; only overtakes around ~100-150 chars. The scan is ~3% of the call, so the dropped P1 (replace the scan) would have slowed short text down. Re-prioritize: P1 = gate the always-on abbreviation rule passes (the real ~25 re.sub/call cost), P2 = list-item guards, P3 = drop the per-call lock. Discovery scan is explicitly left alone.
Profiles both engines on identical input and surfaces the operation-level deltas: per-call wall time, regex op counts/time, and top tottime functions side by side. Used to localize the short/medium gap — it shows we run FEWER regex ops than pySBD (124 vs 266/call) yet are ~9% slower locally, pointing at per-op constants and the pure-Python automaton rather than pass count.
Multi-agent + differential-profiling investigation of the short/medium gap vs pySBD. Key finding: post-#72 there is no regression. Measured: - wall-clock: short 0.99x pysbd, medium 0.93x (parity/ahead) - instruction-count proxy (line-events): short 0.93x (we're ahead), down from 1.24x pre-#72 Root cause of the original ~28% CodSpeed gap: it was largely an instruction-count artifact of pure-Python per-char loops (counted in full by cachegrind, ~0 wall-clock), whose #1 contributor — the zero-width scanner (876 of 2787 line-events) — was gated by #72. We also run FEWER regex ops than pySBD (118 vs 266) and our Aho-Corasick is faster than a naive 'in'-loop on short input, so neither pass-count nor discovery was the cause. Rewrites the plan as a 'extend the lead' menu, tiered by improves-both vs CodSpeed-only: Tier 1 (list early-out/guards, resplit+callback gating, always-on abbreviation gating), Tier 2 (Aho-Corasick DFA delta-table + length-gated hybrid, ~2x scan), Tier 3 (glued-dot scanner C-regex, instruction-count only). Adds benchmarks/differential_profile.py.
Three byte-identical optimizations from the latency review (full suite 1970 passed; AC match-set fuzz-verified vs naive ground truth on 20k random texts): - segmenter._find_sentence_start: search from prior_end via the pattern's pos arg instead of slicing original_text[prior_end:] each sentence, which copied the whole remaining text per sentence (O(n^2) over a document). The flexible fallback patterns are anchor-free, so matching at pos is identical to matching the slice. - AhoCorasickAutomaton: collapse the fail links into a DFA delta-table at build() time, so search() is one dict.get per char with no inner fail-walk loop. Same match set; ~1.5x faster discovery (short 11.3->7.6 us, medium 30.5->20.5 us, now beating the naive in-loop through medium). - ListItemReplacer.iterate_alphabet_array: early-return when re.findall finds no markers, skipping the per-call alphabet-index dict build, lowercasing, and filter on list-free text. Net ~3.5-5% wall-clock on short/medium/large segment(), no behavior change.
yisding
changed the title
yisding
changed the title
Three byte-identical char-presence guards that skip phases which can't
match on the common case. Measured ~10.7% faster on realistic large
English prose (71.3 -> 63.6 ms/call, 35 KB / 500 sentences); suite
unchanged (1967 passed).
- ListItemReplacer.scan_lists: skip the two numbered-list finditer scans
when the text has no digit (every numbered-list pattern requires \d,
and the body does int()). This was the largest of the three (~6.7% of
a realistic large call).
- _GluedLowercaseRunOnRegex.sub: skip the per-character Python scan unless
'....[a-z]' is present (a necessary condition for any change). Uses a
FIXED {4} count, not {4,}, to stay linear on adversarial long period
runs ({4,} backtracks O(n^2)).
- Processor.post_process_segments: skip ReinsertEllipsisRules (5 subs per
segment) unless one of its placeholder sentinels [ƪ♟♝☏∮] is present.
Each guard char is required by every alternative of the gated rule's
regex, so skipping is byte-identical.
…ive rescans
search_for_abbreviations_in_string only ever acts on an abbreviation that
occurs at a word boundary *followed by a period*. The Aho-Corasick
automaton was keyed on the bare abbreviation, so on real prose it matched
~23 short abbreviations as substrings of ordinary words ('al' in 'called',
'no' in 'no one', 'st' in 'stood', 'rd' in 'garden', ...), and each match
triggered a full-text match_re.finditer rescan that found nothing — the
single largest cost on realistic text (~24% of a large call).
Keying the automaton on '<abbr>.' instead is a byte-identical pre-filter:
any word-boundary '<abbr>.' occurrence contains the substring '<abbr>.',
so no real occurrence is missed, and abbreviations whose bare form merely
appears inside other words (with no following period) no longer trigger
the finditer.
Measured: realistic large English prose 64.6 -> 38.7 ms/call (-40%);
short -6.7%. Byte-identical: full suite 1967 passed, and a direct
old-vs-new segment() diff over abbreviation-dense en/en_legal inputs is
character-for-character identical.
yisding
changed the title
_evict_profile is documented to make a re-registration 'rebuilt fresh', but it only dropped the cached LanguageProfile — the per-Abbreviation-class Aho-Corasick data in AbbreviationReplacer._data_cache survived. A language class re-registered after its abbreviation list changed kept the stale automaton. Now also evict the _data_cache entry for the rebound class's Abbreviation. Regression test added (red before, green after). Also fuzz-checked 13k adversarial inputs x 26 languages: zero crashes, zero span round-trip violations, clean=True robust, and the documented feed-at-once streaming contract holds.
yisding
marked this pull request as ready for review
June 14, 2026 04:38
The investigation's conclusions are captured in the PR description and the commit history; the standalone plan file is no longer needed.
The '<abbr>.' period pre-filter (eb7a1c2) is searched on text.lower(), but U+0130 'İ' is the only Unicode char whose .lower() changes length ('İ' -> 'i' + U+0307). An abbreviation occurrence ending in 'İ' followed by a period lowers to '...i̇.', so the 'vi.'-style key missed it and the period over-split (e.g. de 'Band vİ. Der Rest folgt.' wrongly split in 2). Keep the bare key for 'i'-ending abbreviations — the original, always byte-identical behavior — and the period pre-filter for all others. Found by adversarial review. Regression test added (red before, green after) plus a guard asserting 'İ' remains the only length-changing lowercase char.
…oc thread-safety invariants - differential_profile._regex_op_summary counted both the module-level re.sub/re.findall wrapper frame and the <method 'sub' of re.Pattern> builtin it delegates to, double-counting uncompiled calls and biasing the op-count comparison against pre-compiled patterns. Count only the Pattern-method calls. Corrected short-input figure: ours 97 vs pysbd 133 ops/call (was reported 118 vs 266); wall-clock unaffected. - Document the load-bearing lock-ordering invariant in _evict_profile and the AhoCorasickAutomaton publish-after-build thread-safety contract (both surfaced by the concurrency review; no behavior change).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A deep, multi-agent latency investigation of the short/medium-vs-pySBD gap, plus the byte-identical optimizations it produced and one correctness fix. CodSpeed (deterministic) measures the branch at +21.5% vs
main— 10 benchmarks improved +16–34% (large, medium, throughput, streaming,ru). Full suite 1968 passed; every change verified byte-identical (Golden Rules + direct old-vs-newsegment()corpus diffs); robustness fuzz-checked over 13k adversarial inputs × 26 languages (0 crashes, 0 round-trip violations).Optimizations (ranked by measured impact)
<abbr>.— the big one.search_for_abbreviations_in_stringonly acts on an abbreviation at a word boundary followed by a period, but the Aho-Corasick automaton was keyed on the bare abbreviation, so on real prose ~23 short abbreviations matched as substrings of ordinary words (alin called,noin no one) and each triggered a full-textfinditerre-scan finding nothing. Keying on<abbr>.is a byte-identical pre-filter. Realistic large prose 64.6 → 38.7 ms (−40%).search()is onedict.get/char (~1.5×, fuzz-verified vs ground truth on 20k texts).scan_lists) — skip the finditer scans with no digit (~6.7% of a realistic large call).....[a-z](fixed{4}to stay linear on adversarial dot-runs).[ƪ♟♝☏∮]is present.pos=cleanup (the latter measured ~no-op, kept as cleanup).Correctness fix
_evict_profilenow also evicts the abbreviation-data cache on language re-registration (it previously left a stale automaton, contradicting its "rebuilt fresh" contract). Regression test added.Harnesses added
benchmarks/phase_profile.py,differential_profile.py,abbr_scan_compare.py.Method note
Every perf number is a best-of-N or the deterministic CodSpeed run — not single-shot deltas. Several theory-promising changes were measured at ~0% (or regressions) on realistic input and dropped:
replace_numbers/_protect_special_tokensguards (~0%), and thereplace_punctuation→str.translateconversion (measured ~6% slower — chainedstr.replace's C memchr-miss beats per-char table lookup over mostly-letter regions).Deferred (separate focused PRs)
_data_cache→ done here; remaining: unify the three quote-merge implementations;en_es_zhper-abbreviation compile caching.https://claude.ai/code/session_01RBqMKEGgvcZEAXLTPP5FiV