ClipForge

Long video in. Forged clips out. Everything runs on your machine.

ClipForge is an open-source (AGPL-3.0) desktop app that takes a YouTube URL or a horizontal video file and forges vertical 9:16 clips with:

• Smart camera — active-speaker-tracked crop paths, smoothed motion, hard cuts on speaker change, punch-ins fired by actual laughter and vocal energy
• Word-accurate captions — multiple styles, karaoke highlighting, prosodic emphasis (loud words get loud styling), [laughs] tags from real laughter detection
• A virality score you can audit — never a bare number: every clip ships with its subscores, which detectors fired, and every adjustment applied. LLM humor scores get discounted when no actual laughter corroborates them.
• Music-type suggestions — an editable genre/mood/energy brief derived from what's being said and how it sounds
• Optional real-outcomes loop — connect your own Instagram (via your own Meta app, no middleman) and the scorer calibrates against how your clips actually perform

Every model — speech recognition, forced alignment, diarization, laughter detection, audio tagging, face detection, active-speaker detection — runs locally. The only network calls are the video download and 2–3 small LLM calls (bring your own Gemini key, or run fully local via Ollama at reduced scoring quality).

Status

Working end to end: hour-long podcast in, rendered/captioned/scored 9:16 clips out, validated on real footage. The Instagram feedback loop ships in-app (sync, clip↔Reel matching, snapshot history, automatic score calibration). Builds are currently unsigned — install from source below.

Runs on macOS (Apple silicon) and Windows 10/11 x64. The Windows path is validated on every push by the windows workflow: env resolve, full test suite, NSIS build, silent install, and a launch of the installed app on a clean VM.

Demo API

A public showcase of the auditable score lives at https://clipforge-production-1c78.up.railway.app (source in web/):

• GET /health — liveness
• POST /audit — paste a short transcript (+ optional laughs, energy, speaker_changes) and get back subscores, adjustments and the full audit trail, exactly like the desktop app's review screen

curl -s -X POST https://clipforge-production-1c78.up.railway.app/audit \
  -H "Content-Type: application/json" \
  -d '{"transcript":"...","laughs":3,"energy":0.8}'

Layout

pipeline/   Python package — the entire processing pipeline + CLI
app/        Tauri v2 desktop shell (React UI, Python sidecar)

Install from source (macOS)

You need four tools: git, Node, Rust, and uv. Then:

git clone https://github.com/<you>/ClipForge.git
cd ClipForge/app
npm install
npx tauri build --bundles app
ditto src-tauri/target/release/bundle/macos/ClipForge.app /Applications/ClipForge.app
open /Applications/ClipForge.app

The app downloads its speech/audio models (~4–5 GB) on first run with a progress UI, and fetches a caption-capable static ffmpeg automatically if the machine has none. Scoring uses your own Gemini API key, or a local Ollama model at reduced scoring quality — onboarding walks through both.

Install from source (Windows)

You need Rust, the Visual Studio Desktop development with C++ build tools, Node, git, and uv (winget install --id astral-sh.uv -e). Then, in PowerShell:

git clone https://github.com/<you>/ClipForge.git
cd ClipForge\app
npm.cmd install
node_modules\.bin\tauri.cmd build --bundles nsis
# run the installer it produces:
Start-Process (Get-ChildItem src-tauri\target\release\bundle\nsis -Filter *-setup.exe).FullName

First run behaves the same as on macOS: models download behind a progress bar, and a caption-capable static ffmpeg is fetched automatically.

Development

# pipeline
cd pipeline && uv sync && uv run pytest
uv run clipforge run "https://www.youtube.com/watch?v=..."

# app
cd app && npm install && npm run tauri dev

License

AGPL-3.0-or-later. Portions adapted from other open-source projects — see VENDORED-LICENSES.md for the full provenance list.