Decode stdin as UTF-8 and sanitize export filenames

[nmbrthirteen]

Two Windows problems that full-length titles make more likely to hit, since em dashes and smart quotes now reach code that previously only ever saw a 40- or 55-character slice.

stdin was never reconfigured

main.py reconfigured stdout and stderr to UTF-8 but not stdin — the one stream that receives the title. Python falls back to the host locale encoding, cp1252 on most Windows installs. U+201D (curly closing quote) encodes to E2 80 9D, and cp1252 leaves 0x9D undefined, so the request failed to decode before it was ever parsed.

Reproduced directly:

UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 29

The encoding could be lost in three places, so fixing one would have left the hole open:

1. main.py / cli.py did not reconfigure stdin. They do now, so the backend is correct however it is spawned.
2. The launcher's nodeEnv() omitted PYTHONIOENCODING and PYTHONUTF8, which only Run() set. So podcli studio was protected while the MCP server path was not — the asymmetry that made this look intermittent.
3. All four Node→Python spawns rebuilt env by hand, and all four set PYTHONUNBUFFERED while omitting the encoding. That duplication is why it was missed; they now share a pythonEnv() helper.

Verified that Go's exec uses last-wins for duplicate env keys, otherwise the nodeEnv() change would silently no-op on a host that already sets PYTHONIOENCODING. Verified reconfigure() is safe on a TTY, since cli.py is interactive.

Export filenames

The DaVinci exporter interpolated the raw title into a path as {args.title.replace(' ', '_')}.fcpxml, which breaks on path separators, on characters Windows rejects, and on titles long enough to exceed the path limit.

clip_generator already had a correct-but-private sanitizer. It is now the shared safe_filename, used by both. Punctuation is dropped rather than substituted, whitespace is re-collapsed afterwards (removing a padded em dash otherwise leaves __), and Windows reserved device names are escaped — Windows rejects CON, NUL, COM1… as a stem whatever the extension, so CON.fcpxml fails just as CON does.

Scope note

davinci_resolve/cli.py is a spike CLI, invoked by hand and documented in its own README. The shipped MCP export (integration.py:93) takes a caller-supplied output_path, which is required in the tool schema, so no production path derives a .fcpxml name from a title. This is hardening, not a user-facing break. The FCPXML body was already safe: emitter.py builds it with ElementTree, which escapes attribute values.

clip_generator's filenames do change, and it is a hot path. Every difference is an improvement — old names carried __ from dropped punctuation and a trailing _, and an empty title produced _short.mp4.

Verification

• Regression test drives main.py with PYTHONIOENCODING=cp1252 and PYTHONUTF8=0 (so a UTF-8-mode host cannot mask the bug) and asserts In the end — your “shit” has to work round-trips intact.
• 434 Python tests, 61 vitest tests, tsc --noEmit, go build / vet / test all pass.
• tests/test_ai_fallback.py has 3 failures that pre-exist on a clean tree (confirmed by stashing); unrelated to this change.

No release. These reach users on the next tag.

Summary by CodeRabbit

• New Features

  • Added safer, more consistent automatic filename generation for clips and exported files.

• Bug Fixes

  • Improved Windows terminal encoding handling so standard input, output, and error streams use UTF-8 more reliably.
  • Reduced failures when running in locales with non-UTF-8 encodings and when handling titles with special characters or reserved names.

• Tests

  • Added coverage for filename safety and non-ASCII input handling.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1752f639-b00d-499f-89c3-f84fa1b910cd

📥 Commits

Reviewing files that changed from the base of the PR and between 781ed3d and 7a757d9.

📒 Files selected for processing (12)

• backend/cli.py
• backend/main.py
• backend/services/clip_generator.py
• backend/services/integrations/davinci_resolve/cli.py
• backend/utils/text.py
• cli/internal/engine/engine.go
• src/config/paths.ts
• src/services/asset-manager.ts
• src/services/python-executor.ts
• src/ui/web-server.ts
• tests/test_backend_stdin_encoding.py
• tests/test_text_utils.py

---

📝 Walkthrough

Walkthrough

This PR extends UTF-8 stream reconfiguration to include stdin in backend entry points, adds PYTHONIOENCODING/PYTHONUTF8 environment variables in Node-spawned Python processes, introduces a shared pythonEnv() helper adopted across TypeScript subprocess spawns, and adds a safe_filename utility replacing manual filename sanitization in clip generation and DaVinci Resolve CLI output.

Changes

Stdin/UTF-8 Encoding Propagation

Layer / File(s)	Summary
Backend stdin UTF-8 reconfiguration backend/cli.py, backend/main.py	Windows encoding workaround now reconfigures sys.stdin in addition to sys.stdout/sys.stderr.
Node-side Python environment variables cli/internal/engine/engine.go, src/config/paths.ts	nodeEnv() sets PYTHONIOENCODING/PYTHONUTF8; a new pythonEnv() helper builds a UTF-8/unbuffered environment with optional overrides.
pythonEnv adoption in subprocess spawns src/services/asset-manager.ts, src/services/python-executor.ts, src/ui/web-server.ts	yt-dlp, PythonExecutor, download-video endpoint, and runPy() spawn calls now use pythonEnv() instead of manually built environment objects.
cp1252 stdin regression test tests/test_backend_stdin_encoding.py	New test runs backend/main.py with cp1252/non-UTF8 settings and a non-ASCII payload, verifying no UnicodeDecodeError and correct task_id round-trip.

Safe Filename Generation

Layer / File(s)	Summary
safe_filename implementation backend/utils/text.py, tests/test_text_utils.py	Adds FILENAME_MAX, _WINDOWS_RESERVED, and safe_filename() to normalize, filter, truncate, and disambiguate filesystem-safe filenames, with unit tests covering edge cases.
safe_filename adoption in output naming backend/services/clip_generator.py, backend/services/integrations/davinci_resolve/cli.py	Replaces manual filename sanitization with safe_filename() calls when generating clip and .fcpxml output filenames.

Estimated code review effort: 2 (Simple) | ~15 minutes

Sequence Diagram(s)

sequenceDiagram
  participant NodeEngine as Node (engine.go)
  participant PythonExecutor as pythonEnv()/PythonExecutor
  participant Backend as backend/main.py

  NodeEngine->>NodeEngine: nodeEnv() sets PYTHONIOENCODING, PYTHONUTF8
  NodeEngine->>PythonExecutor: spawn(python, env=pythonEnv())
  PythonExecutor->>Backend: send JSON task via stdin (UTF-8)
  Backend->>Backend: reconfigure stdin/stdout/stderr to UTF-8
  Backend-->>PythonExecutor: JSON response via stdout
  PythonExecutor-->>NodeEngine: parsed result

Loading

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/utf8-stdin-and-filename-sanitize

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}