Skip to content

fix(benchmarks): complete local setup and CI scope - #140

Merged
tulayha merged 2 commits into
mainfrom
fix/promptfoo-media-runtime
Sep 1, 2026
Merged

fix(benchmarks): complete local setup and CI scope#140
tulayha merged 2 commits into
mainfrom
fix/promptfoo-media-runtime

Conversation

@tulayha

@tulayha tulayha commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • initialize and verify FFmpeg/ffprobe as part of the one-command Promptfoo setup
  • let VidXP install FFmpeg through a supported package manager when it is missing
  • document the Homebrew prerequisite for a fresh macOS machine
  • classify benchmark-only changes as container- and Desktop-neutral so they do not build product artifacts

Validation

  • npm --prefix benchmarks/codex-mcp run test:setup (5 passed)
  • uv run --no-sync vidxp init --yes --json
  • uv run --no-sync python -m unittest tests/test_ci_scope.py -v (9 passed)
  • uv run --no-sync ruff check utils/ci_scope.py tests/test_ci_scope.py
  • npm exec --yes markdownlint-cli2 -- docs/benchmarking/agent_ablation.md
  • git diff --check

Scope

Internal benchmark and CI tooling only. No storage or model rebuild is required.

@tulayha tulayha changed the title fix(benchmarks): initialize media runtime in setup fix(benchmarks): complete local setup and CI scope Sep 1, 2026
@tulayha
tulayha merged commit 24b916a into main Sep 1, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant