Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/workflows/desktop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,39 @@ concurrency:
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
premiere-packages:
name: Premiere extension packages
runs-on: windows-2025
steps:
- uses: actions/checkout@v7
with:
ref: ${{ inputs.checkout_ref || github.event.inputs.checkout_ref || github.sha }}

- uses: actions/setup-node@v7
with:
node-version: "22"
cache: npm
cache-dependency-path: premiere/package-lock.json

- name: Build and package both Premiere host generations
run: |
npm ci
npm run check
npm run package
working-directory: premiere

- name: Preserve the Premiere packages
uses: actions/upload-artifact@v7
with:
name: vidxp-premiere-packages
path: premiere/packages/*
if-no-files-found: error
retention-days: ${{ inputs.artifact_retention_days || 14 }}

build:
name: ${{ matrix.target == 'windows' && 'Windows x86-64 NSIS' || matrix.target == 'macos' && 'macOS Apple Silicon DMG' || 'Linux x86-64 AppImage' }}
runs-on: ${{ matrix.target == 'windows' && 'windows-2025' || matrix.target == 'macos' && 'macos-15' || 'ubuntu-24.04' }}
needs: premiere-packages
strategy:
fail-fast: false
matrix:
Expand All @@ -75,6 +105,11 @@ jobs:
with:
ref: ${{ inputs.checkout_ref || github.event.inputs.checkout_ref || github.sha }}

- uses: actions/download-artifact@v8
with:
name: vidxp-premiere-packages
path: premiere/packages/

- uses: actions/download-artifact@v8
if: inputs.package_artifact_name != ''
with:
Expand Down
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@ pnpm-debug.log*
/target
/desktop/node_modules/
/desktop/dist/
/premiere/dist/
/premiere/ccx/
/premiere/coverage/
/premiere/packages/
/premiere/.rnd
/desktop/src-tauri/target/
/desktop/src-tauri/gen/
/desktop/src-tauri/binaries/uv-*
Expand Down
11 changes: 11 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"version": "0.2.0",
"configurations": [
{
"type": "uxp",
"request": "attach",
"name": "Attach to VidXP Premiere UXP",
"manifestPath": "${workspaceFolder}/premiere/dist/manifest.json"
}
]
}
84 changes: 81 additions & 3 deletions INSTALLATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,16 @@ Approximate model downloads are:

| Feature | Download |
|---|---:|
| Dialogue search | 2.64 GiB |
| Sound event search | 0.94 GiB |
| Dialogue search | 2.63 GiB |
| Sound event search | 0.91 GiB |
| Scene search | 1.43 GiB |
| Action search | 0.93 GiB |
| Actor matching | 37 MiB |

The Desktop-managed runtime can use about 3 GiB in addition to selected models.
A complete local setup uses about 8.1 GiB before adding videos and indexes.
All five model sets total about 5.95 GiB, so a complete local setup uses about
8.95 GiB before temporary installation space, videos, and indexes. Desktop
calculates the storage plan from the features currently selected during setup.

## Desktop app

Expand Down Expand Up @@ -78,6 +80,13 @@ setup leaves the previous working setup available.
- **AI assistant integration** lets a local MCP-compatible assistant use VidXP.
- **App integration service** lets other local applications use the HTTP API or
Streamable HTTP MCP.
- **Premiere Pro extension** installs the matching Adobe panel and automatically
includes local video processing plus the private app integration service.

The [Premiere Pro extension](docs/integrations/premiere-pro.md) searches media
that is already loaded in an editing project. Desktop includes both supported
Adobe extension packages; users do not need the source repository or a
JavaScript toolchain.

Choose where models should be stored, then decide whether to download them
during setup. VidXP displays the required downloads before starting them.
Expand All @@ -94,6 +103,37 @@ After setup finishes:
Browser and API sharing are off by default. If you enable sharing, use it only
on a trusted local network and follow the warning shown by Desktop.

### Install the Premiere Pro extension

On Windows, VidXP supports Premiere Pro 23.0 or newer with two packaged host
variants:

- Premiere Pro 23.0 through 25.5 uses the CEP extension;
- Premiere Pro 25.6 or newer uses the UXP extension.

In Desktop setup, select **Premiere Pro extension** and the search features you
want. Desktop automatically includes local video processing and its private app
service, then detects standard Premiere installations and installs the matching
package through Adobe Creative Cloud's plugin installer. If Adobe requires
confirmation or elevation, finish the Creative Cloud prompt. Restart Premiere
after installation.

For an existing Desktop-managed VidXP installation, select **Set up Premiere**
on the summary screen. Desktop opens the same setup flow with the Premiere
requirements already selected. **Install for Premiere** remains available there
for reinstalling or retrying only the Adobe package.

For Premiere Pro 23.2, open the panel from **Window > Extensions (Legacy) >
VidXP Search**. For Premiere Pro 25.6 or newer, use **Window > UXP Plugins >
VidXP Search**. If both Premiere generations are installed, Desktop installs
both packages; their host ranges do not overlap.

Plain loopback HTTP is blocked by Premiere UXP on macOS. Desktop therefore does
not claim the 25.6+ macOS panel as usable until the integration has a trusted
local HTTPS transport. The CEP package can use its native local transport on
supported macOS Premiere versions, but still requires host validation before a
release claims support.

## Command line

Command-line installation uses
Expand Down Expand Up @@ -410,6 +450,44 @@ vidxp doctor

Review the suggested package-manager command before approving it.

### Windows Desktop is stuck while checking FFmpeg

Select **Cancel setup**. VidXP stops the package-manager process and keeps the
previous installation and your setup choices. If an older Desktop release does
not show that button, use **Quit VidXP** from the system tray. If it does not
respond, end **VidXP Desktop** in Task Manager. End `winget.exe` there too if it
continues running after VidXP has closed.

Open a new PowerShell window so it receives any PATH changes made by WinGet,
then check whether FFmpeg finished installing:

```powershell
winget list --id Gyan.FFmpeg --exact
where.exe ffmpeg
where.exe ffprobe
```

If WinGet lists `Gyan.FFmpeg` and both executables are found, reopen VidXP
Desktop and retry setup. If the package is missing, or WinGet lists it but the
executables are still not found, repair the package:

```powershell
winget install --id Gyan.FFmpeg --exact --source winget --force --silent `
--disable-interactivity --accept-package-agreements `
--accept-source-agreements
```

Open another new PowerShell window and verify the result:

```powershell
ffmpeg -hide_banner -version
ffprobe -hide_banner -version
ffmpeg -hide_banner -encoders | findstr /i "libx264 aac"
```

The final command should list both `libx264` and `aac`. Reopen VidXP Desktop
and retry setup after all three commands succeed.

### Linux or Windows starts downloading NVIDIA packages

Reinstall with the supported CPU dependency set:
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,7 @@ ordinary MCP tools.

- [Python, HTTP, and MCP installation](INSTALLATION_GUIDE.md)
- [Local HTTP API and MCP server](docs/local-api.md)
- [Premiere Pro extension](docs/integrations/premiere-pro.md)
- [ChatGPT and Codex plugin integration](docs/integrations/openai-plugin.md)
- [Optional capability packages](INSTALLATION_GUIDE.md#optional-dependency-extras)
- [Coolify server setup](docs/deployment/coolify.md)
Expand Down
85 changes: 85 additions & 0 deletions desktop/capability-catalog.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
{
"schema_version": 1,
"capabilities": {
"dialogue": {
"extra": "dialogue",
"modality": "dialogue",
"label": "Dialogue search",
"description": "Index and search spoken dialogue.",
"models": [
{
"cache_key": "models--Qwen--Qwen3-Embedding-0.6B/snapshots/97b0c614be4d77ee51c0cef4e5f07c00f9eb65b3/model.safetensors",
"download_size_bytes": 1207489041
},
{
"cache_key": "models--dropbox-dash--faster-whisper-large-v3-turbo/snapshots/0a363e9161cbc7ed1431c9597a8ceaf0c4f78fcf/model.bin",
"download_size_bytes": 1621668947
}
]
},
"sound": {
"extra": "sound",
"modality": "sound",
"label": "Sound event search",
"description": "Index and search music, environmental sounds, and audio events.",
"models": [
{
"cache_key": "finelap-tokenizer/config.json",
"download_size_bytes": 481
},
{
"cache_key": "finelap-tokenizer/merges.txt",
"download_size_bytes": 456318
},
{
"cache_key": "finelap-tokenizer/vocab.json",
"download_size_bytes": 898823
},
{
"cache_key": "models--AndreasXi--FineLAP/snapshots/b419aa22947d29907a5567f21b81bf3b39a40449/model.safetensors",
"download_size_bytes": 980404741
}
]
},
"scene": {
"extra": "scene",
"modality": "scene",
"label": "Visual scene search",
"description": "Index and search visual scenes.",
"models": [
{
"cache_key": "models--google--siglip2-base-patch16-224/snapshots/75de2d55ec2d0b4efc50b3e9ad70dba96a7b2fa2/model.safetensors",
"download_size_bytes": 1539458338
}
]
},
"actor": {
"extra": "actor",
"modality": "actor",
"label": "Actor recognition",
"description": "Index, inspect, and render actor clusters.",
"models": [
{
"cache_key": "opencv-zoo/face_detection_yunet_2026may.onnx",
"download_size_bytes": 229738
},
{
"cache_key": "opencv-zoo/face_recognition_sface_2021dec.onnx",
"download_size_bytes": 38696353
}
]
},
"videoprism": {
"extra": "videoprism",
"modality": "videoprism",
"label": "Temporal video search",
"description": "Index and search temporal video clips with VideoPrism.",
"models": [
{
"cache_key": "models--google--videoprism-lvt-base-f16r288/snapshots/fb6de9f0eb7bc285be86bdca1cf7daa3e3ef51ff/model.safetensors",
"download_size_bytes": 993993146
}
]
}
}
}
25 changes: 2 additions & 23 deletions desktop/runtime-manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
"dependency_index": "https://pypi.org/simple",
"python_version": "3.14.6",
"uv_version": "0.12.0",
"managed_runtime_estimated_size_bytes": 3221225472,
"surfaces": {
"worker": {
"extra": "local-worker",
Expand All @@ -28,32 +29,10 @@
"server": {
"extra": "server",
"label": "App integration service",
"description": "Run an API and network-style MCP connection for other software. It is private by default and can be shared on your local network with bearer-token authentication.",
"description": "Connect other apps through VidXP's private local API. Local-network sharing is optional and requires bearer-token authentication.",
"default": false
}
},
"capabilities": {
"dialogue": {
"extra": "dialogue",
"modality": "dialogue",
"label": "Dialogue search"
},
"scene": {
"extra": "scene",
"modality": "scene",
"label": "Visual scene search"
},
"actor": {
"extra": "actor",
"modality": "actor",
"label": "Actor recognition"
},
"videoprism": {
"extra": "videoprism",
"modality": "videoprism",
"label": "Temporal video search"
}
},
"media_runtime": {
"strategy": "system",
"executables": [
Expand Down
32 changes: 24 additions & 8 deletions desktop/scripts/model-catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,21 @@
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "src"))

from vidxp.local_probe import desktop_model_cache_catalog # noqa: E402
from vidxp.local_probe import ( # noqa: E402
desktop_capability_catalog,
desktop_model_cache_catalog,
)


CATALOG_PATH = ROOT / "desktop" / "model-cache-catalog.json"
CATALOGS = {
ROOT / "desktop" / "capability-catalog.json": desktop_capability_catalog,
ROOT / "desktop" / "model-cache-catalog.json": desktop_model_cache_catalog,
}


def rendered_catalog() -> str:
def rendered_catalog(value: object) -> str:
return json.dumps(
desktop_model_cache_catalog(),
value,
indent=2,
ensure_ascii=False,
) + "\n"
Expand All @@ -29,13 +35,23 @@ def main() -> int:
mode.add_argument("--write", action="store_true")
mode.add_argument("--check", action="store_true")
arguments = parser.parse_args()
derived = rendered_catalog()
if arguments.write:
CATALOG_PATH.write_text(derived, encoding="utf-8", newline="\n")
for path, derive in CATALOGS.items():
path.write_text(
rendered_catalog(derive()),
encoding="utf-8",
newline="\n",
)
return 0
if CATALOG_PATH.read_text(encoding="utf-8") != derived:
stale = [
path.relative_to(ROOT).as_posix()
for path, derive in CATALOGS.items()
if not path.exists()
or path.read_text(encoding="utf-8") != rendered_catalog(derive())
]
if stale:
raise SystemExit(
"desktop/model-cache-catalog.json is stale; run npm run "
f"{', '.join(stale)} is stale; run npm run "
"model-catalog:write from desktop/."
)
return 0
Expand Down
Loading
Loading