fix: Windows-native compatibility for the derive-reorganization branch - #63
Merged
Conversation
The daemon lifecycle code in internal/bridge used Unix-only syscall.Setpgid and group SIGKILL, breaking the native Windows build of internal/bridge, internal/worker and cmd/kaas; the Python entrypoint also registered SIGPIPE unconditionally, which Windows Python does not provide. - Split the process-group setup into daemon_unix.go and add daemon_windows.go, which tree-kills via taskkill /T /F and falls back to Process.Signal when the process already exited. - Guard the SIGPIPE registration with hasattr(signal, "SIGPIPE"). With these, `go run ./cmd/kaas -f etc/kaas-dev.toml` plus `pnpm dev` run the full stack on Windows; `make dev` keeps working on Unix. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The daemon's wire protocol is UTF-8 JSON lines, but a piped Python
process on Windows decodes stdin with the ANSI code page (cp936 on
Chinese Windows) and errors='surrogateescape'. The UTF-8 bytes the Go
bridge writes became mojibake laced with lone surrogates, which then
failed the strict UTF-8 encode of the LLM request:
extract: bridge: AI engine error INTERNAL_ERROR: UnicodeEncodeError:
'utf-8' codec can't encode character '\udcac': surrogates not allowed
- server_daemon.main() now reconfigures stdin/stdout/stderr to UTF-8
before serving (streams without reconfigure(), like StringIO in
tests, are left alone).
- KBStore built rel_path via str(relative_to(...)), which yields
backslash separators on Windows, so the CLI route keyed extractions
as raw\a.md while the worker route keyed them raw/a.md. Both sites
now use Path.as_posix(), restoring the byte-identical-extraction
contract between the two ingestion routes.
- The parity test read extraction files with the locale codec; it now
reads them as UTF-8 explicitly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Python text IO without an explicit encoding follows the OS locale: UTF-8 on the Linux CI, but cp936 (GBK) on Chinese Windows. On that platform the engine wrote GBK bytes for master-index.md, topic indexes, timeline, classify cache, compile state, article writes, people stubs, the compile log and derived manifests, which then broke every strict UTF-8 reader downstream — including the Go bridge and the web UI — and failed ~110 py tests natively. - Add encoding="utf-8" to every bare read_text/write_text/open text call in py/src and py/tests. - KBStore.extraction_rel_path built paths with str(Path(...)), which yields backslash separators on Windows; use a slash join so the layer's rel-path keys match the Go/daemon form everywhere. - copy_documents now rejects POSIX-style absolute rel_paths too — Windows pathlib treats "/etc/passwd" as relative, so the previous is_absolute() guard let it slip through to the escape check and surface as a ValueError instead of the specified DeriveError. - test_ask_uses_kb_dir compares resolved forms (resolve() rewrites a POSIX-style root to a drive path on Windows); the trailing-newline list_derived test is skipped on NTFS, which forbids such names. py suite on zh-CN Windows: 107 failing -> 3 (all three are a pre-existing intermittent write-phase race that also reproduces without these changes). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
These two test files only exist on this branch (added with the derive-reorganization merge), so the locale sweep on fix/windows-native-build could not cover them. Their fixtures wrote master-index and article files with the cp936 default, which broke once the product side started reading and writing strict UTF-8. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Under a parallel-extract workload on Windows, Path.resolve() intermittently returns a \?\-prefixed verbatim path when another thread creates a parent directory (e.g. extraction/) at the same moment. The verbatim prefix then fails every is_relative_to(base) comparison, so KBStore._resolve raised "path escapes kb_dir" for files inside the KB and compile lost random documents (reproducible: ~3 in 12 suite runs failed on different tests each time). _resolve and resolve_kb_dir now strip the \?\ and \?\UNC\ prefixes before comparing. A 8-thread stress loop over _resolve with a concurrent mkdir went from intermittent escapes to zero, and the flaky compile test files went from 3-5 failures per 12 runs to 0. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
tybot02
approved these changes
Sep 9, 2026
tybot02
merged commit Sep 9, 2026
7594157
into
bybit-exchange:feat/derive-knowledge-reorganization
10 checks passed
4 tasks
lucasmaan
pushed a commit
that referenced
this pull request
Sep 11, 2026
…ation (#64) * fix(upload): skip unsupported files inside ZIP instead of rejecting entire archive Previously, if a ZIP contained any file with an extension not in allowedInnerExtensions (e.g. .jpg, .png), the entire ZIP was rejected with "extension not allowed". This made it impossible to upload ZIPs containing a mix of documents and non-document files. Now unsupported entries are silently skipped and only supported files are extracted and processed. * fix(derive): hash-based fallback slug for pure CJK topics slugFromTopic (Go) and normalise_slug (Python) now produce a deterministic "t-{sha256[:10]}" slug when the regex normalization strips every character (e.g. pure Chinese topics like '定价'). Previously these topics were rejected with 'invalid slug'. * fix(worker): don't burn retry attempts on circuit breaker rejection Two bugs caused a cascading failure where 126 out of 136 failed tasks were killed by 'breaker open' without ever reaching the LLM: 1. Extract phase treated circuit.ErrOpen as a regular failure, calling Nack and burning a retry attempt. The pipeline phase already handled this correctly by abandoning without Nack. Apply the same pattern to extract: abandon and let RecoverExpired requeue the task. 2. Dispatcher claimed maxConc tasks per tick in half-open state, but the breaker only allows one in-flight probe — the rest were instantly rejected. Limit claiming to 1 task when the breaker is half-open. Root cause: transient LLM outage (timeouts + connection errors) tripped the breaker; the 30s cooldown/half-open cycle then permanently killed queued tasks every cycle by burning all 3 retry attempts on instant ErrOpen rejections. * docs(plan): add technical plan - derive knowledge reorganization * feat(derive): add data structures and entity index builder (Feature: p1-feat-001) * feat(classify): add topic-aware classify prompt and function (Feature: p1-feat-004) * feat(derive): add entity name resolution with LLM canonicalization (Feature: p1-feat-002) * feat(derive): add aggregation planning and reorganize orchestrator (Feature: p1-feat-003) * feat(derive): wire reorganize into compile_kb and derive_kb (Feature: p1-feat-005) * docs(plan): add technical plan - 2026-09-07-derive-split-oversized-merge * feat(merge): add budget estimation helpers (Feature: p2-feat-001) * feat(compile): add batch packing and iterative merge (Feature: p2-feat-002) * feat(compile): wire batch splitting into _process_article (Feature: p2-feat-003) * test(compile): add batch splitting tests (Feature: p2-feat-004) * docs(plan): add technical plan - 2026-09-07-derive-sub-article-split * feat(compile): add sub-article threshold and helpers (Feature: p3-feat-001) * feat(compile): add sub-article support to _merge_batch_split (Feature: p3-feat-002) * feat(compile): wire sub-article splitting into _process_article (Feature: p3-feat-003) * test(compile): add sub-article splitting tests (Feature: p3-feat-004) * fix(compile): cap per-batch budget to prevent LLM timeout (Feature: p4-feat-001) * fix(compile): lower _MAX_BATCH_BUDGET to 20K to prevent merge-path timeouts * feat(derive): add reorganize flag and derive-flow diagrams - .gitignore: exclude .ph directory - server_daemon: hardcode reorganize=True in _handle_derive for validation - docs/assets: add derive-flow SVG diagrams (en + zh) * docs(plan): add technical plan - 20260908-derive-reorganize-config-v2 * feat(daemon): read reorganize from payload instead of hardcoding (Feature: p5-feat-003) * feat(config): add DeriveConf with reorganize setting (Feature: p5-feat-001) * feat(derive): thread reorganize config through bridge and runner (Feature: p5-feat-002) * test(daemon): add reorganize parameter tests for _handle_derive (Feature: p5-feat-005) * test(derive): add Go tests for reorganize config, bridge, and runner (Feature: p5-feat-004) * docs(plan): normalize date format to yyyy-mm-dd * docs(plan): add technical plan - 2026-09-09-batch-parallel-write * feat(compile): add batch-parallel infrastructure (Feature: p6-feat-001) Add _ChainResult dataclass, _get_batch_parallel_sem() semaphore factory, _sem_guard context manager, and _pre_split_chains chunking function. These are the building blocks for parallelizing batch-level LLM calls within _merge_batch_split. The semaphore follows the exact pattern of _get_section_sem() in merge.py: reads KB_BATCH_PARALLEL_MAX_CONCURRENT per call, warns once on invalid values, caches keyed on bound. Tests cover semaphore lifecycle (default, env override, invalid values), _sem_guard acquire/release/noop/exception-safety, _pre_split_chains chunking correctness, order preservation, and edge cases. * feat(compile): implement _run_chain for batch-parallel chains (Feature: p6-feat-002) * feat(compile): refactor _merge_batch_split to three-phase parallel structure (Feature: p6-feat-003) * test(compile): add comprehensive batch-parallel tests (Feature: p6-feat-004) * fix(derive): auto-deduplicate slug on conflict instead of erroring When normalise_slug generates a slug from a pure-CJK topic (or any topic that produces a deterministic hash-based slug like t-e999a859f8), re-running the same topic would hit SlugExistsError. The user had to manually pass --force or --slug to retry. Now, when the slug is auto-generated (not user-provided) and force=False, derive_kb() automatically appends -2, -3, etc. until a free slug is found. User-provided slugs still error on conflict (existing behavior preserved). Python: _deduplicate_slug() in _layout.py, wired into derive_kb(). Go: deduplicateSlug() in derive.go for the HTTP API handler. Tests: 7 new tests covering no-conflict, single/double conflict, truncation, validation, user-slug preservation, and Go-side dedup. * docs(plan): add technical plan - 2026-09-09-parallel-reorg * feat(reorganize): add aggregate-plan-feedback prompt template (Feature: p7-feat-002) * feat(compile): add batch_parallel_threshold for serial fallback (Feature: p7-feat-001) * feat(reorganize): add two-round feedback to plan_aggregation (Feature: p7-feat-003) * docs(plan): add technical plan - 2026-09-09-builds-page-redesign * feat(i18n): add Builds page i18n strings (Feature: p8-feat-007) * feat(web): extend derived API client and add build types (Feature: p8-feat-002) * feat(api): add ListDerivedJobsPaged and DeleteDerivedJob endpoints (Feature: p8-feat-001) * feat(web): add shared build components (useAutoPolling, StatusStageBadge, StatsBar) (Feature: p8-feat-003) * feat(web): add DeriveJobsTab and DeriveJobDetailDialog components (Feature: p8-feat-005) * feat(web): extract TasksTab and TaskDetailDialog components (Feature: p8-feat-004) * feat(web): add Builds page shell, routing, nav, and simplify DeriveDialog (Feature: p8-feat-006) * fix(web): adjust StatsBar tab sizing and DeriveJobsTab column widths * fix(web): set appropriate column widths for Builds tables (Feature: col-widths) * fix(web): widen Attempts column (Feature: attempts-width) * fix(web): make StatsBar tabs inline instead of full-width * fix(web): add divider and padding around StatsBar * fix(web): horizontal layout for tabs and search bar with divider * docs(plan): add technical plan - 2026-09-09-write-phase-early-parallel * fix(web): remove divider from filter bar * feat(compile): add batch_parallel_batch_limit config and lower default concurrency (Feature: p9-feat-001) * feat(compile): add batch-count early break to Phase A and _run_chain (Feature: p9-feat-002) * fix: Windows-native compatibility for the derive-reorganization branch (#63) * fix: build and run the dev stack natively on Windows The daemon lifecycle code in internal/bridge used Unix-only syscall.Setpgid and group SIGKILL, breaking the native Windows build of internal/bridge, internal/worker and cmd/kaas; the Python entrypoint also registered SIGPIPE unconditionally, which Windows Python does not provide. - Split the process-group setup into daemon_unix.go and add daemon_windows.go, which tree-kills via taskkill /T /F and falls back to Process.Signal when the process already exited. - Guard the SIGPIPE registration with hasattr(signal, "SIGPIPE"). With these, `go run ./cmd/kaas -f etc/kaas-dev.toml` plus `pnpm dev` run the full stack on Windows; `make dev` keeps working on Unix. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: keep CJK payloads intact when running natively on Windows The daemon's wire protocol is UTF-8 JSON lines, but a piped Python process on Windows decodes stdin with the ANSI code page (cp936 on Chinese Windows) and errors='surrogateescape'. The UTF-8 bytes the Go bridge writes became mojibake laced with lone surrogates, which then failed the strict UTF-8 encode of the LLM request: extract: bridge: AI engine error INTERNAL_ERROR: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcac': surrogates not allowed - server_daemon.main() now reconfigures stdin/stdout/stderr to UTF-8 before serving (streams without reconfigure(), like StringIO in tests, are left alone). - KBStore built rel_path via str(relative_to(...)), which yields backslash separators on Windows, so the CLI route keyed extractions as raw\a.md while the worker route keyed them raw/a.md. Both sites now use Path.as_posix(), restoring the byte-identical-extraction contract between the two ingestion routes. - The parity test read extraction files with the locale codec; it now reads them as UTF-8 explicitly. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: make every KB text file UTF-8 regardless of OS locale Python text IO without an explicit encoding follows the OS locale: UTF-8 on the Linux CI, but cp936 (GBK) on Chinese Windows. On that platform the engine wrote GBK bytes for master-index.md, topic indexes, timeline, classify cache, compile state, article writes, people stubs, the compile log and derived manifests, which then broke every strict UTF-8 reader downstream — including the Go bridge and the web UI — and failed ~110 py tests natively. - Add encoding="utf-8" to every bare read_text/write_text/open text call in py/src and py/tests. - KBStore.extraction_rel_path built paths with str(Path(...)), which yields backslash separators on Windows; use a slash join so the layer's rel-path keys match the Go/daemon form everywhere. - copy_documents now rejects POSIX-style absolute rel_paths too — Windows pathlib treats "/etc/passwd" as relative, so the previous is_absolute() guard let it slip through to the escape check and surface as a ValueError instead of the specified DeriveError. - test_ask_uses_kb_dir compares resolved forms (resolve() rewrites a POSIX-style root to a drive path on Windows); the trailing-newline list_derived test is skipped on NTFS, which forbids such names. py suite on zh-CN Windows: 107 failing -> 3 (all three are a pre-existing intermittent write-phase race that also reproduces without these changes). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test: read/write derive-branch test fixtures as UTF-8 These two test files only exist on this branch (added with the derive-reorganization merge), so the locale sweep on fix/windows-native-build could not cover them. Their fixtures wrote master-index and article files with the cp936 default, which broke once the product side started reading and writing strict UTF-8. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: stop Windows resolve() races from faking kb_dir escapes Under a parallel-extract workload on Windows, Path.resolve() intermittently returns a \?\-prefixed verbatim path when another thread creates a parent directory (e.g. extraction/) at the same moment. The verbatim prefix then fails every is_relative_to(base) comparison, so KBStore._resolve raised "path escapes kb_dir" for files inside the KB and compile lost random documents (reproducible: ~3 in 12 suite runs failed on different tests each time). _resolve and resolve_kb_dir now strip the \?\ and \?\UNC\ prefixes before comparing. A 8-thread stress loop over _resolve with a concurrent mkdir went from intermittent escapes to zero, and the flaky compile test files went from 3-5 failures per 12 runs to 0. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * docs(plan): add technical plan - 2026-09-09-builds-master-detail.md * feat(api): add AbortSignal support to getTask (Feature: p10-feat-002) * feat(i18n): add Sheet description entries for builds detail (Feature: p10-feat-003) * feat(ui): extend Sheet component with close button and SheetDescription (Feature: p10-feat-001) * feat(builds): create TaskDetailSheet and DeriveJobDetailSheet (Feature: p10-feat-004) * feat(builds): refactor tabs to use Sheet detail with clickable rows (Feature: p10-feat-005) * refactor(builds): remove obsolete Dialog detail components (Feature: p10-feat-006) * test(builds): update tests for Sheet components and clickable rows (Feature: p10-feat-007) * docs(plan): add technical plan - 2025-09-10-builds-tab-refactor.md * feat(ui): update StatusStageBadge for partial status and optional stage (Feature: p11-feat-008) * feat(i18n): add build job keys and rename tab labels (Feature: p11-feat-009) * feat(api): create build jobs API client (Feature: p11-feat-007) * feat(store): add BuildJob model and BuildJobStore interface (Feature: p11-feat-001) * feat(ui): create BuildJobDetailSheet component (Feature: p11-feat-011) * feat(ui): create BuildJobsTab component (Feature: p11-feat-010) * feat(store): implement SQLite build_jobs table and store methods (Feature: p11-feat-002) * test(store): add build_jobs store tests (Feature: p11-feat-003) * feat(api): add build job API handlers and routes (Feature: p11-feat-004) * feat(ui): update Builds page to use BuildJobsTab (Feature: p11-feat-012) - Replace TasksTab import/usage with BuildJobsTab in Builds.tsx - Replace listTasks stats calls with listBuildJobs for Normal tab counts - Delete TasksTab.tsx and its test file - Update Builds.test.tsx to mock listBuildJobs instead of listTasks - Fix test assertions to match current i18n tab labels (Normal, Derive Topic) * feat(api): create BuildJob on submit (Feature: p11-feat-005) * test(builds): update all frontend tests for tab refactor (Feature: p11-feat-013) - StatusStageBadge.test.tsx: add partial status amber badge tests, optional stage tests - BuildJobsTab.test.tsx: new test file covering table rendering, sort, search, pagination, delete confirmation, detail sheet, partial in filter/delete - BuildJobDetailSheet.test.tsx: new test file covering job detail display, task list rendering, nested TaskDetailSheet, error indicator - BuildJobDetailSheet.tsx: guard against undefined tasks in optimistic display - StatsBar.test.tsx, Builds.test.tsx: already updated by prior features - TasksTab.test.tsx: already deleted by prior features * test(api): add build job API tests and update submit tests (Feature: p11-feat-006) * fix: enable TOC sidebar scrolling when content overflows Add overflow-y-auto to the TOC aside so long tables of contents can scroll independently. Remove the ineffective sticky positioning on the inner wrapper since the aside itself is the scroll container. * docs(plan): add technical plan - 2026-09-10-chat-kb-switcher * feat(ui): update Session type, API functions, and KB store (Feature: p12-feat-005) * feat(store): add KBSlug to Session model and update SQLite schema (Feature: p12-feat-001) * test(store): update session tests for kb_slug filter (Feature: p12-feat-002) * feat(ui): create ChatKBSelector component (Feature: p12-feat-006) * test(api): update session tests for kb_slug support (Feature: p12-feat-004) * feat(ui): update SessionList and Chat.tsx for KB-scoped sessions (Feature: p12-feat-007) * test(ui): update frontend tests for KB switcher (Feature: p12-feat-008) * docs(plan): add technical plan - 2026-09-10-derive-quality-fix.md * feat(prompts): add language directives to 6 prompts (Feature: p13-feat-004) * feat(derive): implement multi-round voting in select_by_topic (Feature: p13-feat-001) * test(merge): add regression test for _create_system language directive (Feature: p13-feat-005) * test(derive): add multi-round voting test suite (Feature: p13-feat-003) * feat(derive): propagate filter_rounds through derive_kb, CLI, and daemon (Feature: p13-feat-002) * fix: make classify tokenizer CJK-aware with character bigrams _title_words() stripped all CJK characters, causing _relevance_score() to return 0 for every Chinese title and dedup_create_new() to skip all Chinese titles. Together these made the classifier create one wiki article per source file instead of merging related content. Add _tokenize() that produces character bigrams for CJK runs (standard IR technique, comparable granularity to Latin words) and splits Latin runs on non-alphanumeric boundaries. _title_words() and the inline topic tokenization in _relevance_score() both delegate to it. Side fix: hyphenated topic tags like 'sun-wukong' now split into {'sun', 'wukong'} instead of collapsing to {'sunwukong'}, so they match article titles correctly. * fix(derive): wire filter_rounds from TOML config through Go to Python daemon * test(derive): add filter stability test script from diagnosis * docs(plan): add technical plan - 2026-09-10-cr-fixes * style(api): fix import grouping in session.go (Feature: p14-feat-002) * fix(api): replace log.Printf with structured slog (Feature: p14-feat-001) * refactor(ui): remove unused kbSlug prop from SessionList (Feature: p14-feat-005) * refactor(ui): extract shared formatDate utility (Feature: p14-feat-003) * refactor(store): extract duplicated status CASE/WHEN SQL to const (Feature: p14-feat-006) * refactor(api): embed buildJobDTO in buildJobDetailDTO (Feature: p14-feat-004) * refactor(store): extract generic paged query builder (Feature: p14-feat-007) * refactor(ui): extract shared PagedJobTable component (Feature: p14-feat-008) * docs(plan): add technical plan - 2026-09-10-derive-drawer-article-tasks.md * feat(derive): make filter voting threshold configurable via filter_threshold Add filter_threshold (float64, range (0,1]) to the full config chain: TOML [derive] section, KAAS_DERIVE_FILTER_THRESHOLD env override, Go bridge DeriveRequest, Python CLI --filter-threshold, and the select_by_topic engine function. When set, the acceptance threshold becomes ceil(filter_rounds * filter_threshold). When unset (0), the existing supermajority formula ceil(2N/3) is preserved, so the change is fully backward-compatible. Examples with filter_rounds=5: filter_threshold=0.5 → threshold=3 (simple majority) filter_threshold=0.67 → threshold=4 (supermajority, current default) filter_threshold=1.0 → threshold=5 (unanimity) * feat(i18n): add derive article table keys (Feature: p15-feat-001) * feat(ui): create DeriveWikiPreviewSheet component (Feature: p15-feat-002) * test(ui): add DeriveWikiPreviewSheet tests (Feature: p15-feat-004) * feat(ui): add article table to DeriveJobDetailSheet (Feature: p15-feat-003) * test(ui): add DeriveJobDetailSheet article table tests (Feature: p15-feat-005) * docs(plan): add UI polish plan - 2026-09-10-builds-ui-polish.md * style(ui): unify column widths to 160px (Feature: p16-feat-002) * feat(ui): add immediate thinking indicator in chat (Feature: p16-feat-005) * style(ui): fix tab label, badge spacing, sheet widths (Feature: p16-feat-004) * feat(ui): replace row click with details button (Feature: p16-feat-003) * feat(ui): restyle status badges with WCAG-compliant colors (Feature: p16-feat-001) --------- Co-authored-by: 王大虎 <157195705@qq.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cherry-picks the Windows-native compatibility fixes (originally developed on
fix/windows-native-build, merged into the localtestingbranch) on top offeat/derive-knowledge-reorganization, so this branch also builds and runscleanly on a zh-CN Windows machine. All five commits apply without conflicts
and the full suite is green natively.
(
syscall.Setpgid, group SIGKILL) intodaemon_unix.go/daemon_windows.go(taskkill tree-kill), and guard the SIGPIPEregistration in the Python entrypoint.
stdin with the ANSI code page (cp936) + surrogateescape, corrupting the
Go bridge's UTF-8 JSON into lone surrogates, which then fail the strict
UTF-8 encode of the LLM request (
extract: ... UnicodeEncodeError ... '\udcac' ... surrogates not allowed).server_daemon.main()now pinsstdin/stdout/stderr to UTF-8.
encoding="utf-8"on all bare text IO inpy/srcandpy/tests(master/topic indexes, timeline, caches, compile state, people stubs,
compile log, manifests, test fixtures). Without this, files land on disk
as GBK on Chinese Windows and every strict UTF-8 reader downstream
(including the Go bridge and the web UI) breaks.
Path.resolve()races from faking kb_dir escapes —under parallel extraction,
resolve()intermittently returns\?\-prefixed verbatim paths while another thread mkdirs a parentdirectory, which fails
is_relative_to(base)and makes compile droprandom documents (~3-12% of suite runs failed on rotating tests).
_strip_verbatim()now removes the prefix before containment checks(
KBStore._resolveandresolve_kb_dir).Test plan
(one skip is NTFS-forbidden newline dir names)
go build ./...andgo test ./internal/bridge/pass natively on WindowsKBStore._resolvewith concurrent mkdir:escapes 0 (intermittent before)
Co-Authored-By: Claude Opus 4.7 noreply@anthropic.com