DLSS5 Infinity Studio

DLSS5 Infinity Studio is an experimental Windows video-processing workstation that applies per-frame monocular depth estimation and NVIDIA DLSS Neural Rendering to offline video. A native C++ host provides a polished WebView2 UI, while a separate GPU worker decodes, processes, and encodes the video without putting the interface at risk if a codec, model, or GPU provider fails.

Important

This is an independent research project. It is not an NVIDIA product and is not affiliated with or endorsed by NVIDIA. NVIDIA runtime files, neural model weights, and FFmpeg are deliberately not included in this repository.

What it does

The application accepts a local video, reconstructs a depth guide for every frame using DirectML, submits matching color/depth/motion resources to a local signed NVIDIA Streamline DLSS Neural Rendering runtime, and encodes the result at a selected output resolution. Compatible audio and subtitle streams can be copied from the source.

Current capabilities include:

• Native Windows C++ application with a local WebView2 interface
• Native source and destination file dialogs
• Separate dlss5infy-cli GPU worker with JSON progress reporting
• FFprobe media inspection and FFmpeg raw-BGRA decoding
• Persistent D3D12 and DirectML Depth Anything V2 inference
• Signed Streamline feature-1004 / NGX feature-18 evaluation when configured
• 720p, 1080p, and 4K output presets
• HEVC NVENC, H.264 NVENC, and CPU x265 encoding choices
• Optional preservation of compatible audio and subtitle streams
• Frame-accurate source-range selection using start time and duration
• User-configurable feature-1004 style, preset, performance, tone, structure, intensity, masking, and depth-orientation controls
• Bounded one-, two-, or four-frame pipeline modes
• Release staging and local runtime configuration scripts

Processing pipeline

Local video
    |
    v
FFprobe inspection
    |
    v
FFmpeg decode (BGRA) ---> bounded frame queue
                              |
                              v
                    DirectML depth inference
                              |
                              v
                 DLSS Neural Rendering evaluation
                              |
                              v
                       bounded output queue
                              |
                              v
                    FFmpeg / NVENC encoding
                              |
                              v
                    Processed video + copied media

What “four frames in flight” means

Four frames in flight does not apply DLSS four times to one image. Every source frame is evaluated once. Up to four reusable CPU frame slots allow FFmpeg decoding, ordered GPU processing, and FFmpeg encoding to overlap. DLSS submissions remain in source-frame order for temporal correctness.

Requirements

To run

• Windows 10 or Windows 11 x64
• An NVIDIA RTX GPU with a compatible current driver
• Microsoft Edge WebView2 Runtime
• FFmpeg and FFprobe available on PATH
• A compatible Depth Anything V2 ONNX model supplied by the user
• Compatible signed NVIDIA Streamline/NGX runtime files supplied by the user

The runtime configuration script validates and copies the expected local components. It does not download or bypass licensing for any NVIDIA component or model.

To build

• Visual Studio 2022 or later with Desktop development with C++
• CMake 3.28 or later
• Windows SDK with D3D12 support
• WebView2 SDK (restored by CMake)
• Windows App SDK ML / DirectML dependencies (restored by CMake)
• Optional: a compatible Streamline SDK tree for feature-1004 integration

Build from source

Configure and compile the UI and worker:

cmake -S . -B build -A x64
cmake --build build --config Release

To compile with the signed feature-1004 bridge enabled:

cmake -S . -B build -A x64 `
  -DINFY_STREAMLINE_SDK_ROOT=C:/path/to/streamline `
  -DINFY_NVIDIA_RUNTIME_DIR=C:/path/to/signed/runtime

cmake --build build --config Release

Build output is written to build/bin/Release.

Configure local runtime files

Supply your own licensed runtime files and model:

.\scripts\Configure-Runtime.ps1 `
  -BinaryDirectory .\build\bin\Release `
  -Model C:\path\to\depth-anything-v2-small-fp16.onnx `
  -NvidiaRuntime C:\path\to\signed\nvidia\runtime

The script expects compatible copies of:

• sl.interposer.dll
• sl.common.dll
• sl.dlss_nr.dll
• nvngx_dlssnr.dll
• depth-anything-v2-small-fp16.onnx

Do not commit these files. The repository ignores DLLs, ONNX weights, local models, build output, packaged releases, and sample videos.

Using the Studio

1. Start DLSS5 Infinity Studio.exe.
2. Choose a local source video.
3. Choose an output location.
4. Select 720p, 1080p, or 4K output.
5. Optionally enter a start time and duration using HH:MM:SS.mmm or seconds. Leave duration empty to process from the selected start through the end.
6. Adjust the NVIDIA DLSS Neural Rendering controls or retain their defaults.
7. Select the encoder and quality preset.
8. Choose the number of frames in flight. Four is recommended for throughput.
9. Enable audio or subtitle preservation when required.
10. Start the neural render and monitor phase, frame, and progress information.

The advanced DLSS panel exposes only values accepted by the current local feature-1004 bridge: neural style 0–6, render preset 0–7, performance modes 1/2/3/4/6, intensity 0–1, local and skin structure -1–1, local tone -1–1, global tone 0–1, auto mask, and depth inversion. Reset defaults restores the known working configuration.

The source aspect ratio must be preserved. Cinematic sources such as 1280×534 should scale to a matching cinematic frame rather than being stretched to 1920×1080.

Command-line worker

The Studio launches the worker automatically, but it can also be exercised directly:

.\dlss5infy-cli.exe `
  --input C:\video\input.mp4 `
  --output C:\video\output.mp4 `
  --resolution 1920x1080 `
  --codec hevc_nvenc `
  --crf 22 `
  --inflight 4 `
  --start 90 `
  --duration 15 `
  --dlss-intensity 1 `
  --dlss-style 2 `
  --dlss-render-preset 0 `
  --dlss-performance-mode 6 `
  --preserve-audio `
  --json-progress

Use --model, --runtime, --ffmpeg, and --ffprobe when those components are not located beside the worker or on PATH.

Verified native smoke test

The native pipeline has completed an end-to-end test using a two-second, 30 FPS source:

Check	Result
Decoded and evaluated frames	60
Output	1280×720 HEVC NVENC
Encoded video frames	60
Duration	2.0 seconds
Audio	Original AAC stream preserved
Worker exit code	0

The test exercised DirectML depth inference, signature verification of the configured Streamline runtime, feature-1004 evaluation, NVENC output, and audio preservation.

Create a local binary package

.\scripts\Build-Release.ps1 -Version 0.1.0

The ZIP is written under dist. It intentionally excludes NVIDIA binaries, model weights, FFmpeg, and test media. End users configure those components locally with Configure-Runtime.ps1.

Current research limitations

This is a working native MVP, not a finished production encoder. Important areas still under development include:

• The present motion guide is a same-resolution zero-motion resource inherited from the initial camera prototype; production temporal reconstruction needs optical flow or another accurate motion-vector source.
• Scene-cut detection and explicit temporal-history resets are still needed.
• Variable-frame-rate timestamp preservation needs broader validation.
• HDR, 10-bit formats, and complete color-metadata propagation are not yet implemented.
• Resumable jobs, crash recovery, persistent queues, previews, and detailed ETA telemetry are future work.
• Model/runtime compatibility is dependent on the locally supplied versions.

Please do not characterize experimental output as guaranteed DLSS quality or as an NVIDIA-certified result.

Test media and copyright

Only process media you own or have permission to modify. Sample films, model weights, and third-party binaries are not part of this repository.

For real-person technical testing, the Blender Foundation's Tears of Steel is available separately under CC BY 3.0. Its actors retain portrait and privacy rights, so follow Blender's usage guidance and do not imply endorsement.

Repository and dependency licensing

Original project source is available under the MIT License. Dependencies and interoperated runtimes retain their own licenses. Read THIRD_PARTY_NOTICES.md before distributing a build.

In particular:

• NVIDIA Streamline, NGX, DLSS, and signed runtime files remain subject to NVIDIA's terms and are not redistributed here.
• FFmpeg licensing depends on the exact build configuration and enabled codecs.
• Model weights remain subject to their publisher's license.
• Microsoft WebView2, DirectML, and ONNX Runtime retain their applicable terms.

Contributing

Focused issues and pull requests are welcome. Useful contribution areas are optical flow, scene-cut handling, aspect-ratio preservation, HDR/10-bit media, timestamp correctness, encoder recovery, and reproducible quality evaluation.

When reporting a processing problem, include the source codec, dimensions, frame rate, output preset, encoder, GPU and driver, FFmpeg version, and the worker's JSON progress/error output. Do not attach copyrighted source media or proprietary runtime DLLs.

Author

Created by SamG-Coder.