Skip to content

windows: scan-deps rule wrapped in cmd /c hits the 8191-char limit — use clang-scan-deps -o instead of shell redirection #261

Description

@Sunrisepeak

Summary

On Windows the clang scan-deps ninja rule is wrapped in cmd /c purely to get shell redirection (> $out). cmd.exe caps a command line at 8191 characters, so any package whose include-dir list is large fails with

The command line is too long.
error: build failed

clang-scan-deps has supported -o <file> since LLVM 17 (verified on the llvm xpkgs mcpp itself installs — 20.1.7 and 22.1.8), so the redirect — and with it the cmd /c wrapper and its 8191 ceiling — is unnecessary. Dropping it puts the rule back on ninja's direct CreateProcess path, whose limit is 32767.

Where

src/build/ninja_backend.cppm:665-673:

if constexpr (mcpp::platform::is_windows) {
    // Wrap in cmd /c for shell redirection (ninja on Windows uses
    // CreateProcess which doesn't interpret > as redirect).
    append("  command = cmd /c \"$scan_deps -format=p1689 -- "
           "$cxx $local_includes $cxxflags $unit_cxxflags -c $in -o $compile_target > $out\"\n");
} else {
    append("  command = $scan_deps -format=p1689 -- "
           "$cxx $local_includes $cxxflags $unit_cxxflags -c $in -o $compile_target > $out\n");
}

Proposed:

append("  command = $scan_deps -format=p1689 -o $out -- "
       "$cxx $local_includes $cxxflags $unit_cxxflags -c $in -o $compile_target\n");

(one branch for both platforms). If a wider compiler-version floor is a concern, the belt-and-braces alternative is ninja's rspfile/rspfile_content for $local_includes $cxxflags, which removes the length ceiling regardless of how the output is produced.

Reproduction / evidence

Sunrisepeak/opencv-m v0.0.7 (vendored full OpenCV 5 source build) consumed as a dependency in mcpp-index's opencv-module* workspace members, workspace (windows) leg of mcpp-index CI:

  • measured scan-deps command lengths: 8370 chars (base profile), 8570 chars (dnn feature) — both over 8191, all six scan invocations failed
  • 48 -I entries: 26 from the package's [build].include_dirs, ~13 private per-OS gen/ dirs emitted by its build.mcpp, ~11 from the transitive compat.ffmpeg
  • the failing TU is the package's own src/videoio.cppm, not consumer code

The decisive factor is the path prefix, which is why this only shows up on the consumer side:

context package root length
opencv-m's own CI D:\a\opencv-m\opencv-m\ 23
consumed as a dep D:\a\mcpp-index\mcpp-index\tests/examples/opencv-module-dnn\.mcpp\.xlings\data\xpkgs\mcpplibs-x-opencv\0.0.7\opencv-m-0.0.7\ 124

The same package builds and tests green on the windows leg of its own CI (5-leg matrix, all green) and fails as a dependency solely because ~100 extra characters per -I push the total past the cmd.exe limit. Any sufficiently deep consumer path reproduces it; a package needs only ~60 include dirs to hit the ceiling even at a short path.

Environment: mcpp 0.0.101, windows-x86_64, llvm@20.1.7 (mcpp's own default on that platform).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions