ci: add CodSpeed latency benchmarks (baseline infra) - #70
Merged
Merged
Conversation
Add deterministic instruction-count benchmarks (pytest-codspeed) for the latency-sensitive paths — segment(), should_wait_for_more(), and StreamSegmenter.feed() — plus a CodSpeed workflow that runs them on push and pull_request and posts per-PR regression/speedup deltas. CodSpeed counts CPU instructions under instrumentation, so results are stable on noisy shared GitHub runners (a real algorithmic regression like the prior streaming double-segmentation shows up as an instruction delta, not jitter). The benchmark tooling is isolated in a new 'bench' dependency group and a dedicated benchmarks/test_latency_codspeed.py; the main test matrix targets tests/ only, so it never collects the file or installs the plugin. Requires a maintainer to install the CodSpeed GitHub App and add a CODSPEED_TOKEN secret; without it the job still runs but skips the upload.
Grant id-token: write and drop the token input from the CodSpeed action. The action exchanges GitHub's short-lived OIDC token for upload credentials, so no CODSPEED_TOKEN secret is required — only the CodSpeed GitHub App install.
Cover non-Latin scripts so CodSpeed tracks regressions the English samples would miss: zh exercises the whitespace-free CJK resplit path and ru exercises the abbreviation/resplit rules over a non-ASCII alphabet.
Congrats! CodSpeed is installed 🎉
You will start to see performance impacts in the reports once the benchmarks are run from your default branch.
|
yisding
marked this pull request as ready for review
June 14, 2026 00:56
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0dd081f62b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A plain 'uv run' implicitly re-syncs the default dev group before pytest, pulling unrelated dev tooling (mypy, nltk, hypothesis, ...) back into the benchmark environment and running install work under CodSpeed instrumentation. --no-sync runs in the exact bench-only environment the prior 'uv sync --no-default-groups --group bench' step created. Verified empirically: after a plain 'uv run', mypy reappears in the env; with --no-sync it stays absent and all 10 benchmarks still run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Stand up the CodSpeed latency-benchmark infrastructure on its own, off
main, so it establishes a pre-optimization baseline. The performance work in #69 can then be measured as real deltas against that baseline once #69 rebases on the updatedmain.Contains only the benchmark infra — no library code changes:
benchmarks/test_latency_codspeed.py—pytest-codspeedbenchmarks for the latency paths:segment()(short/medium/large),should_wait_for_more(),StreamSegmenter.feed()(conservative/aggressive), andsegment()for zh (CJK resplit path) and ru (Cyrillic abbreviation/resplit path). 10 benchmarks total..github/workflows/codspeed.yml— runs them on push tomainand on PRs, authenticated via OIDC (id-token: write, noCODSPEED_TOKENsecret).pyproject.toml/uv.lock— isolatedbenchdependency group (pytest+pytest-codspeed), separate fromdevso the main test/lint matrix is untouched.Why separate from #69
CodSpeed compares each commit against the default branch. If the benchmarks first landed inside #69, the baseline would already include the optimizations and the gains would be invisible. Landing this on
mainfirst captures the slow baseline; #69 then shows the speedups.Isolation / safety
tests/only, so it never collects this file or installspytest-codspeed.main(segment,should_wait_for_more,StreamSegmenter), so they run identically pre- and post-optimization.Setup
One-time: install the CodSpeed GitHub App on the repo (already done — the bot is commenting). OIDC means no secret to manage.
https://claude.ai/code/session_01RBqMKEGgvcZEAXLTPP5FiV
Generated by Claude Code