Integrate the live bg-letter line into the input-safety audit - #11
Merged
Conversation
Two live regressions on v0.6.1, both rooted in contracts that predate full-word composing (b3a2cb2): 1. Accept keys (Tab/Space/Enter) could not accept a visible completion. The Tab-accept branch kept the pre-composing contract "locked buffer => prefix already in the app, commit the ghost suffix only", but full-word composing holds the ENTIRE word in the preedit regardless of lock state — so Tab dropped the typed prefix once the buffer locked, and Space/Enter had no accept semantics at all (the unique- prediction auto-commit is effectively dead with multi-candidate predictions). Accept now commits the full composing word whenever a dual buffer is active (suffix-only remains for the non-dual keyval path, where the prefix was already forwarded), the logic is shared via accept_ghost_completion(), and Space/Return accept a visible completion while still forwarding their delimiter. An anticipatory ghost (nothing typed) is deliberately NOT committed by Space/Return. 2. Backspace could not delete committed text while any preedit was showing. The adapter unconditionally consumed a backspace press on preedit_was_active, swallowing core-*forwarded* backspaces under an anticipatory next-word ghost (and desyncing the keyval-path word mirror). The adapter now honors the core's forward/consume verdict: composing-edit backspace is still consumed (the core emits no ForwardKey there), forwarded backspace reaches the app. Replay telemetry gains space_accept/enter_accept reasons so boundary accepts are no longer logged as word_boundary rejections. Receipts: cargo test -p smartkey-core: 390 lib + 20 integration green; ibus/test_accept_backspace.py: 8 green (engine->_execute_actions->IBus wiring with contract-faithful fake core, both key paths). All new regression tests verified RED against the reverted semantics. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TScyQBAQeK8n4m3jfGehFk
Escape's hypothesis-phase arm won before the ghost check, so dismissing a visible completion in an unlocked composing word reset_word()'d the whole preedit and silently discarded the typed prefix; in the locked case the preedit was hidden whole, leaving the typed word invisible but still pending. Either way there was no safe "I want my word plus a literal space" path once a completion was showing. Escape now dismisses ONLY the completion: the composing preedit is re-shown with the typed word alone, and the candidate list is cleared so the rejected prediction cannot be re-committed by the aggressive unique-match Space path. Escape then Space/Return commits the typed word verbatim. No-ghost behaviors are unchanged (hypothesis cancel, forward otherwise). Receipts: cargo test -p smartkey-core 393 lib + 34 integration green; pytest ibus/test_accept_backspace.py 9 green (adds the Escape→Space literal-space flow through do_process_key_event→_execute_actions). The hypothesis word-loss regression test verified RED against the old hypothesis-first arm. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TScyQBAQeK8n4m3jfGehFk
Live check falsified the Space-accept: with Bulgarian typing, Space committed a wrong (English) prediction, injecting words the user never typed. Operator decision: Tab is the ONLY accept key. Reverted BOTH auto-accept paths on the word boundary — they are the same injection class (landing a word the user did not type): - the Space/Return boundary-accept of the visible completion (added in 0a7bd1b, now removed); - the legacy "aggressive" unique-candidate ReplaceWord, which could commit last_predictions[0] even though it was never SHOWN, breaking "accept == last displayed prediction". Space/Return commit exactly the TYPED word again. Kept from 0a7bd1b/ 83224b4: backspace honors the core verdict, Tab commits the full composing word (dual_buffer.is_some() gate), literal-space Escape. Replay telemetry reverted accordingly: a Space/Return typed-word commit logs as a word_boundary rejection, not an acceptance. New regression tests: Space/Return with a visible completion commit the typed word only; the unshown unique candidate is never auto-committed; Tab emits exactly one CommitText equal to the last shown composition (anti-double + anti-desync). These assert the exact opposite of the 83224b4 tests they replace, so they are RED against the reverted code by construction. Receipts: cargo test -p smartkey-core 394 lib + 34 integration green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TScyQBAQeK8n4m3jfGehFk
Per-keystroke JSONL trace of the live decision path (spec 2026-07-12): key_in -> core verdict (consume/forward + dual-buffer/lock/hypothesis via new debug_state() FFI getter + predictions) -> dispatched actions -> accept (committed vs typed + script class) -> outcome. One grouped seq per keystroke, recorded at the REAL do_process_key_event -> _execute_actions seam. Levels: OFF by default (single bool guard, zero I/O, production byte-for-byte identical); SMARTKEY_DEBUG=1 structural (no verbatim content — script classes, lengths, verdicts, state only); SMARTKEY_DEBUG=full adds verbatim content with a stderr banner. The lang class is always logged: committed_lang=lat vs typed_lang=cyr is the Space-inject signature, visible even in structural mode. The legacy predictions.log/replay.jsonl content logs are now gated on =full for the same reason. Privacy guardrails: output only under ~/.local/state/smartkey/debug/ (dir 0700, files 0600); a SMARTKEY_DEBUG_DIR override inside the repo is refused; .gitignore safety net for trace/dump files; 48h age purge + 50MB size cap; ring-buffered with flush off the hot path (every 50 events + focus_out/reset/disable); smartkey-debug CLI (enable/disable/status/dump/tail/wipe) + DEBUG.md repro recipe. Receipts: pytest 23 green (real-seam integration, OFF strict no-op, structural-no-content + full-content, in-repo dir refusal, 0600/0700 modes, purge/wipe, emit overhead budget: OFF <5µs, on <100µs); cargo 394 lib + 34 integration green; clippy 0 warnings; ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TScyQBAQeK8n4m3jfGehFk
The Codex cross-brand re-gate correctly flagged that emit() flushed the ring to disk every FLUSH_EVERY events from inside do_process_key_event, and that the legacy full-content logs used line buffering plus explicit per-write flush — synchronous I/O in exactly the path the trace is meant to diagnose. emit() now NEVER touches the file system: it appends to the in-memory ring and, past the threshold, requests ONE deferred flush through an injected scheduler (the engine passes GLib.idle_add, so the write runs after the key event completes). Without a scheduler the ring is written only by the explicit idle-point flushes (focus-out/reset/disable); deque(maxlen) still bounds memory. Legacy predictions.log/replay.jsonl switch to block buffering with no per-write flush. New tests pin the invariant: emit past 4x the threshold leaves the trace file at 0 bytes until flush; the threshold schedules exactly one deferred flush (GLib idle contract) and re-arms only after it ran. Receipts: pytest 25/25 green; cargo 394 lib green; ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TScyQBAQeK8n4m3jfGehFk
At SMARTKEY_DEBUG=1 the key_in event logged keyname=chr(keyval) plus the raw keyval/keycode for EVERY key, so the whole typed keystream was verbatim reconstructible — "structural, no typed text" was false (ADVOCATE demonstrated live: typing at level 1 reconstructed the word and its keyvals). key_in now applies a printable-key privacy rule: a printable key is logged only as key_class=char + key_lang (script class — still enough to catch the Space-inject signature); raw keyname/keyval/keycode are reserved for special keys (Tab/BackSpace/Escape/...) or level=full. The accept event applies the same rule to its trigger-key field (composing-guard commits can fire on printable digits/punctuation). Also removes the dead "space_autocommit" acceptance telemetry (ADVOCATE m4): with the aggressive path gone, a Space replace only means a post-commit caps/translit correction of the typed word and must not be logged as an acceptance. DEBUG.md wording updated. New tests: driving real printable keyvals at structural asserts no keyname/keyval/keycode and no quoted chars land in the file; special keys keep their names; level=full keeps printable keynames. Receipts: pytest 27/27 green; cargo 394 lib green; ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TScyQBAQeK8n4m3jfGehFk
main.py raised logging.DEBUG for ANY non-empty SMARTKEY_DEBUG (including "0"/"off"), and the engine's debug log lines carry verbatim action payloads (committed/ghost text) into smartkey.log — a third content sink outside the trace discipline, with default permissions and no retention, not covered by smartkey-debug wipe. logging.DEBUG is now gated on SMARTKEY_DEBUG == "full" only, matching the legacy content logs: "=1" (structural) stays content-free end to end. Also corrects the stale "idle points flush too" comment on the legacy log buffering (ADVOCATE minor). Receipts: pytest 27/27 green; ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TScyQBAQeK8n4m3jfGehFk
…un (S4) Live trace (364 key events, full level) exposed two coupled S4 defects: one Tab committed "колко на" — the F8 multi-word extension chained the predicted NEXT word into the ghost, so the accept payload carried an unasked extra word — and the anticipatory next-word ghost re-armed instantly after the accept, so four consecutive Tabs committed "на" four times (the most frequent word is always predicted next). Fixes: 1. KAB removal of the F8 next-word chain: compute_ghost_suffix returns exactly the completion of the CURRENT word, so display == commit == one word (preserves the anti-desync invariant). The try_multi_word_extension helper is kept dead-code-annotated with the labelled revival condition (word-granular multi-word UX). 2. post_commit_pipeline gains show_anticipatory: accept paths pass false — after a Tab-accept the user must type again before a new prediction is offered. Typing-boundary commits (Space/Return, digit and punctuation guards) keep the v0.6.1 anticipatory behavior. No timing guard needed: a second Tab has no ghost and is a plain forward (design decision per dispatch 5c17abf38fd4). Tests: tab-accept-does-not-rearm (verified RED against the reverted pipeline flag — the anticipatory ghost fires at confidence 0.349 vs gate 0.20); ghost-never-chains behavioral guard + Space keeps its anticipatory pipeline (scope fence). Honesty note: the chain half cannot be RED-falsified in the unit fixture — ensemble confidence saturates ~0.35 below the const MULTI_WORD_MIN_CONFIDENCE=0.40 gate (probe receipts in review thread); reintroduction is guarded by the call-site removal + dead-code annotation and the live-trace evidence. Receipts: cargo 397 lib + 34 integration green; pytest 27/27; clippy 0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TScyQBAQeK8n4m3jfGehFk
PR #10 (8ef48dc) branched from 23c11f4 and never picked up 5045811 "fix(input): preserve raw Bulgarian words at boundaries" — the commit behind the native module actually installed on the operator's machine. Everything else the live line carries had already been re-landed through PR #7 (d3ef2c5), so the genuine delta is small and specific: * post_commit_autocorrect as an explicit opt-in (struct field, Default, both JSON parsers, and the gate around pipeline steps 1-2) * the lossless-transliteration guard in try_language_correction * three regression tests, including bg_punctuation_position_letters_commit_exact_words ONE GENUINE EITHER/OR, resolved deliberately. The show_anticipatory argument at the three typing-boundary call sites: the live line passes true (47acd8a, "typing boundaries keep v0.6.1 behavior"), PR #7 passes false (d3ef2c5, "a fresh preedit races the forwarded delimiter — GTK/Electron/browser clients apply Space/Return to the new ghost"). It is one value and cannot be both. Taken: false, because it is the later decision on this exact point, it is the only value under which BOTH sides' tests pass, and its failure mode is a missing next-word suggestion rather than duplicated typing in a browser. Consequence worth stating plainly: with 5045811's autocorrect gate default-off AND show_anticipatory=false, post_commit_pipeline returns no actions on the default config at every typing boundary. Both sides chose that independently; together it makes the post-commit pipeline inert by default. One duplicate test helper removed to compile: press_raw_with was defined by both sides, byte-equivalent modulo a parameter name (E0428). Gates against the merged tree, not against either parent: cargo fmt --check clean cargo clippy --all-targets clean cargo test -p smartkey-core --lib 433 passed, 1 ignored pytest ibus/ 122 passed The IBus suite needs a native module built from THIS tree; the installed .so predates both lines and exposes neither debug_state nor current_word, which is why the combined native/adapter test fails against it. Built out of tree and run against it: green. No live artifact was replaced. Six adversarial verifiers each tried to prove a named fix had been lost — keystream privacy, Tab/Escape/accept dispatch, browser duplication, bg-letter, cancel semantics, ContentType stickiness. All six failed to, with hunk-level and AST-level evidence rather than keyword matching. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Po7GZbvPSVjr16bSBBmhA
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Pull request overview
This PR merges the remaining “live line” deltas into the input-safety audit work by hardening the IBus adapter against sensitive-field leakage and phantom key storms, while aligning the Rust core’s boundary/auto-correct behavior and adding regression tests + CI coverage to prevent cross-worktree seam regressions.
Changes:
- Add IBus content-type–based sensitive-input kill switch (no core calls, no trace, no surrounding-text reads) plus cancel-path safety filtering in the Python adapter.
- Improve handling of modifier/Caps Lock semantics and tighten multiple core behaviors (post-commit autocorrect gating, transliteration losslessness, fuzzy/punctuation performance guards, backspace lock behavior).
- Add extensive regression tests (Python adapter safety suite + new Rust tests) and run the native IBus suite in CI via a checkout-built PyO3 module.
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| ibus/test_input_safety.py | New adapter-policy and cross-language seam regression suite for sensitive fields, phantom keys, and cancel semantics. |
| ibus/smartkey_engine.py | Implements sensitive-input policy, phantom key consumption, cancel-path filtering, and related state bookkeeping. |
| DEBUG.md | Documents shipped keystroke diagnostics tool, levels, and privacy guardrails. |
| crates/smartkey-py/src/lib.rs | Maps IBus Caps Lock latch bit into core modifiers. |
| crates/smartkey-core/src/input.rs | Adds CAPS_LOCK modifier semantics, post-commit autocorrect opt-in, lossless transliteration guard, preedit flush adjustments, and multiple regression tests. |
| crates/smartkey-core/src/eval.rs | Exposes latency window sample count for structural (non-stopwatch) perf assertions in tests. |
| crates/smartkey-core/src/ensemble.rs | Folds case for trie scoring and prevents fuzzy fallback for prefixes within edit budget (plus tests). |
| .gitignore | Stops ignoring DEBUG.md so user-facing debug tool docs ship. |
| .github/workflows/ci.yml | Builds current checkout’s native module and runs pytest ibus/ in CI to validate the real adapter↔native seam. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1104
to
+1111
| # The core reporting one of these means the word it was holding is no | ||
| # longer in flight: every ``HideGhost`` it emits follows a ``reset_word()`` | ||
| # or a commit, and ``CommitText``/``ReplaceWord`` resolve the word by | ||
| # definition. ``forward`` is deliberately absent — the core emits a bare | ||
| # ForwardKey for a CTRL chord or a release while the dual buffer still | ||
| # holds the word. | ||
| _COMPOSITION_RESOLVING_ACTIONS = frozenset({"commit", "replace", "hide"}) | ||
|
|
RMANOV
pushed a commit
that referenced
this pull request
Aug 3, 2026
Found by Copilot's review of #11. The code is on master via #10. `_COMPOSITION_RESOLVING_ACTIONS` counted `hide`, on the reasoning that every `HideGhost` follows a `reset_word()` or a commit. It does not: the core emits a bare `HideGhost` whenever the prefix has no ghost suffix. Measured against the real native module, because the two core entry points differ and only one is affected: process_keycode('x','q','z') -> ['composing'] each; counter 1,2,3 handle_key('x','q','z') -> ['hide','forward'] each, current_word() 'x','xq','xqz' So the dual-buffer path was always fine and the KEYVAL COMPAT path was not — the branch taken when the client sends `keycode == 0` or the installed native module predates `process_keycode`. There, every letter of an out-of-vocabulary word zeroed `_keys_since_content_type`, so `_composition_in_flight()` reported "nothing open" while the core still held the word, and a content type arriving mid-word was not escalated to sensitive. `hide` is still honoured, but only once the core confirms the word is gone (`current_word()` empty). When the core cannot be asked — an older native module, or a fake in a test — the answer is False: leaving the counter running keeps the fail-safe armed. A false "still composing" costs one content-type change treated as sensitive; a false "resolved" costs the kill switch not firing inside a password field. Three regression tests, two of which fail against the pre-fix behaviour under a mutation run. The third (a hide after the word is gone still resolves) pins the opposite direction so the fix cannot pin the counter on forever and make SmartKey inert for the rest of the field. Worth recording: the first version of the mid-word test asserted `_composition_in_flight()` and passed under mutation, because that helper ORs in `_preedit_active`. It now asserts the counter directly, and drives keycode=0 — the only path where the defect is reachable. cargo fmt clean, clippy clean, cargo test 432 passed, pytest ibus/ 125. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014Po7GZbvPSVjr16bSBBmhA
Resolves the single conflict in try_language_correction. Both sides had independently rewritten the merge-base predicate `other_word.len() < 2` into the same char-count + equal-length test; master's form hoists `other_chars` and reuses the `typed_chars` binding already introduced at the top of the function, so it is taken verbatim. No behavioural choice was involved — the two predicates are equivalent. Adds post_commit_autocorrect_gate_is_load_bearing_both_ways. The gate on steps 1-2 is the highest-value line either side contributes, and nothing was verifying it: forcing it to `if true` (restoring the live "чака" -> "aka" corruption) or to `if false` (deleting the post-commit subsystem) both left the suite green. The losslessness guard cannot cover this, because "цар" -> "car" is a same-length transliteration it accepts. Falsified in both directions before committing: `if true` fails the default-off half, `if false` fails the opt-in half — different asserts, so each half carries its own weight. Verified: 434 passed / 0 failed, clippy clean, fmt clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DtMgpWub58hvEMBjJLzMnn
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #10. Same audit work, plus the one live commit #10 never picked up.
Why this exists
#10 (
8ef48dc) branched from23c11f4and never picked up5045811"fix(input): preserve raw Bulgarian words at boundaries" — the commit behind the
native module actually installed on the operator's machine. Everything else the
live line carries had already been re-landed through #7 (
d3ef2c5), so thegenuine delta is small and specific:
post_commit_autocorrectas an explicit opt-in (struct field,Default, both JSON parsers, and the gate around pipeline steps 1–2)try_language_correctionbg_punctuation_position_letters_commit_exact_wordsOne genuine either/or, resolved deliberately
show_anticipatoryat the three typing-boundary call sites. The live line passestrue(47acd8a, "typing boundaries keep v0.6.1 behavior"); #7 passesfalse(
d3ef2c5, "a fresh preedit races the forwarded delimiter — GTK/Electron/browserclients apply Space/Return to the new ghost"). It is one value and cannot be both.
Taken:
false— it is the later decision on this exact point, it is the onlyvalue under which both sides' tests pass, and its failure mode is a missing
next-word suggestion rather than duplicated typing in a browser.
Consequence worth stating plainly: with
5045811's autocorrect gate default-offand
show_anticipatory=false,post_commit_pipelinereturns no actions on thedefault config at every typing boundary. Both sides chose that independently;
together it makes the post-commit pipeline inert by default.
Gates — run against the merged tree, not against either parent
cargo fmt --checkcargo clippy --all-targetscargo test -p smartkey-core --libpytest ibus/The IBus suite needs a native module built from this tree: the installed
.sopredates both lines and exposes neither
debug_statenorcurrent_word, which iswhy the combined native/adapter test fails against it. Built out of tree and run
against it — green. No live artifact was replaced.
Verification
Six adversarial verifiers each tried to prove a named fix had been lost —
keystream privacy (
8bcfc27/e8e45a1), Tab/Escape/accept dispatch, browserduplication (
08b3fb1), bg-letter (5045811), cancel semantics, ContentTypestickiness. All six failed to, with hunk-level and AST-level evidence rather than
keyword matching.
One duplicate test helper removed to compile:
press_raw_withwas defined by bothsides, byte-equivalent modulo a parameter name (E0428).
🤖 Generated with Claude Code
https://claude.ai/code/session_014Po7GZbvPSVjr16bSBBmhA