SM64 RT64 Path Tracing on Linux

Run Super Mario 64 with real-time path tracing on Linux using Wine, a patched vkd3d-proton, and DXVK.

🎉 Update 2026-04-17: The vkd3d-proton fix has been merged upstream by maintainer Hans-Kristian Arntzen (merge commit dd78b4f). It's now in master. Once the next release (after v3.0b) lands, you won't need to build vkd3d-proton from source — any distro's vkd3d-proton package or Proton/Proton-GE will have it built-in. Until then, this installer still builds from source to get the fix.

This repo provides:

sm64rt — Vanilla SM64 textures + RT64 path tracing
Render96ex RT64 — HD textures + HD models + RT64 path tracing

⚡ Quick Install (One Command, Multi-Distro)

curl -sL https://raw.githubusercontent.com/danielbanariba/sm64rt-linux-guide/main/install.sh | bash -s -- /path/to/baserom.us.z64

Auto-detects your distro and installs dependencies via the right package manager:

Distro family	Detected from	Package manager
Arch (CachyOS, Manjaro, EndeavourOS)	`arch`, `cachyos`, `manjaro`, `endeavouros`	`pacman` + `yay`/`paru`
Debian (Ubuntu, Mint, Pop!_OS)	`debian`, `ubuntu`, `linuxmint`, `pop`	`apt-get`
Fedora (RHEL, CentOS, Rocky, AlmaLinux)	`fedora`, `rhel`, `centos`, `rocky`, `almalinux`	`dnf`
openSUSE (Leap, Tumbleweed)	`opensuse`, `suse`	`zypper`

Or clone and run:

git clone https://github.com/danielbanariba/sm64rt-linux-guide.git
cd sm64rt-linux-guide
./install.sh /path/to/baserom.us.z64

The installer handles everything: deps, SDL2, vkd3d-proton patch + build, sm64rt build, Render96ex RT64 build, Wine prefix, DXVK, VC++ runtime, gamepad DB, launchers. Takes ~20 minutes total. After that:

~/build/sm64rt-linux/run-sm64rt.sh           # vanilla + RT
~/build/sm64rt-linux/run-render96-rtx.sh     # HD textures/models + RT

Optional flags:

SKIP_RENDER96=1 ./install.sh ROM — skip the Render96ex build (faster, smaller)
SKIP_DEPS=1 ./install.sh ROM — skip pacman/yay (if already installed)
BUILD_DIR=/custom/path ./install.sh ROM — install elsewhere

Manual Installation (if you want to understand each step)

Requirements

OS: Arch Linux (tested on CachyOS kernel 6.19.12)
GPU: NVIDIA RTX series (tested on RTX 3090 Ti, driver 595.58)
ROM: baserom.us.z64 — US version of Super Mario 64 (SHA-1: 9bef1128717f958171a4afac3ed78ee2bb4e86ce). You must legally dump this from your own cartridge.

The Problem We Solved

RT64 (the path tracing renderer) calls DispatchRays without explicitly calling SetComputeRootSignature. Native Windows D3D12 drivers handle this implicitly, but vkd3d-proton (the D3D12-to-Vulkan translator used by Wine/Proton) strictly requires it — silently dropping every ray dispatch and producing black frames.

Our fix (✅ merged upstream 2026-04-17): vkd3d-proton PR #2940 — adds implicit root signature binding as a fallback when none is set. Maintainer's review: "Broken application, but guess we need to workaround then <_< sigh." Fair enough.

Step 1: Install Dependencies

# Build tools + MinGW cross-compiler
sudo pacman -S --needed base-devel git python mingw-w64-gcc mingw-w64-binutils \
  mingw-w64-crt mingw-w64-headers mingw-w64-winpthreads make wine winetricks meson ninja glslang

# From AUR
yay -S --needed vkd3d-proton-mingw-git mingw-w64-glew

Step 2: Download SDL2 MinGW Development Libraries

mkdir -p ~/build/sdl2-mingw && cd ~/build/sdl2-mingw
curl -sL -o SDL2-devel-mingw.tar.gz \
  "https://github.com/libsdl-org/SDL/releases/download/release-2.30.10/SDL2-devel-2.30.10-mingw.tar.gz"
tar xzf SDL2-devel-mingw.tar.gz

Fix the sdl2-config script paths — edit SDL2-2.30.10/x86_64-w64-mingw32/bin/sdl2-config:

Change --cflags to output -I${prefix}/include -I${prefix}/include/SDL2 -Dmain=SDL_main
Change --libs to output -L${libdir} -lmingw32 -lSDL2main -lSDL2 -mwindows

Step 3: Build sm64rt

cd ~/build
git clone https://github.com/DarioSamo/sm64rt.git
cp /path/to/baserom.us.z64 sm64rt/baserom.us.z64
cd sm64rt

Apply cross-compilation fixes

Case-sensitive includes — change Windows.h to windows.h in:
- include/rt64/rt64.h
- src/pc/gfx/gfx_rt64_context.h
Missing stdio.h — add #include <stdio.h> after #include <SDL2/SDL.h> in src/pc/audio/audio_sdl2.c

GCC 15 compatibility — add to the Windows CFLAGS in Makefile:

-Wno-error=int-conversion -Wno-error=incompatible-pointer-types -Wno-error=implicit-function-declaration

Compile

make -j$(nproc) VERSION=us RENDER_API=RT64 EXTERNAL_DATA=1 WINDOWS_BUILD=1 \
  CROSS=x86_64-w64-mingw32- CC=x86_64-w64-mingw32-gcc CXX=x86_64-w64-mingw32-g++ \
  LD=x86_64-w64-mingw32-g++ AS=x86_64-w64-mingw32-as \
  OBJCOPY=x86_64-w64-mingw32-objcopy OBJDUMP=x86_64-w64-mingw32-objdump \
  TARGET_ARCH=i386pe TARGET_BITS=64 NO_BZERO_BCOPY=1 \
  SDLCONFIG=$HOME/build/sdl2-mingw/SDL2-2.30.10/x86_64-w64-mingw32/bin/sdl2-config

Copy SDL2.dll to build/us_pc/:

cp ~/build/sdl2-mingw/SDL2-2.30.10/x86_64-w64-mingw32/bin/SDL2.dll build/us_pc/

Step 4: Build Render96ex RT64 (optional, HD textures + models)

cd ~/build
git clone --branch tester_rt64alpha https://github.com/Render96/Render96ex.git render96-rt
cp /path/to/baserom.us.z64 render96-rt/baserom.us.z64
cd render96-rt

Download assets:

git clone --depth 1 --branch models_vanilla https://github.com/Render96/ModelPack.git /tmp/ModelPack
cp -r /tmp/ModelPack/Render96/* actors/
git clone --depth 1 https://github.com/pokeheadroom/RENDER96-HD-TEXTURE-PACK.git /tmp/HDTextures
curl -sL -o /tmp/dynos.7z "https://github.com/Render96/ModelPack/releases/download/3.25/Render96_DynOs_v3.25.7z"

Apply the same fixes as sm64rt (Windows.h, stdio.h, GCC 15 flags), then:

# Build native tools first
CC=gcc CXX=g++ make -C tools -j$(nproc)

# Extract ROM assets
python3 extract_assets.py us

# Cross-compile
make -j$(nproc) VERSION=us RENDER_API=RT64 EXTERNAL_DATA=1 TEXTURE_FIX=1 \
  WINDOWS_BUILD=1 NOEXTRACT=1 CROSS=x86_64-w64-mingw32- \
  CC=x86_64-w64-mingw32-gcc CXX=x86_64-w64-mingw32-g++ \
  LD=x86_64-w64-mingw32-g++ AS=x86_64-w64-mingw32-as \
  OBJCOPY=x86_64-w64-mingw32-objcopy OBJDUMP=x86_64-w64-mingw32-objdump \
  TARGET_ARCH=i386pe TARGET_BITS=64 NO_BZERO_BCOPY=1 \
  SDLCONFIG=$HOME/build/sdl2-mingw/SDL2-2.30.10/x86_64-w64-mingw32/bin/sdl2-config

# Copy runtime files
cp ~/build/sdl2-mingw/SDL2-2.30.10/x86_64-w64-mingw32/bin/SDL2.dll build/us_pc/
cp -r /tmp/HDTextures/gfx build/us_pc/res/
7z x -o"build/us_pc/dynos/packs/" /tmp/dynos.7z -y

Step 5: Build vkd3d-proton from master

The fix is now in vkd3d-proton master (merged via PR #2940 on 2026-04-17). The latest release (v3.0b, Dec 2025) does not yet include it, so for now build from master upstream:

git clone https://github.com/HansKristian-Work/vkd3d-proton.git ~/build/vkd3d-proton
cd ~/build/vkd3d-proton
git submodule update --init --recursive
./package-release.sh master ~/build/vkd3d-output --no-package

Once a new release after v3.0b is published, you can skip this step entirely — just install the distro's vkd3d-proton package (or use Proton/Proton-GE which bundles it).

Step 6: Setup Wine Prefix

export WINEPREFIX=~/.wine-sm64rt

# Create prefix
WINEDLLOVERRIDES="mscoree=" wineboot -u

# Install patched vkd3d-proton
cp ~/build/vkd3d-output/vkd3d-proton-master/x64/d3d12.dll ~/.wine-sm64rt/drive_c/windows/system32/
cp ~/build/vkd3d-output/vkd3d-proton-master/x64/d3d12core.dll ~/.wine-sm64rt/drive_c/windows/system32/

# Install DXVK (for dxgi.dll — required to replace Wine's broken dxgi)
curl -sL -o /tmp/dxvk.tar.gz "https://github.com/doitsujin/dxvk/releases/download/v2.7.1/dxvk-2.7.1.tar.gz"
tar xzf /tmp/dxvk.tar.gz -C /tmp
cp /tmp/dxvk-2.7.1/x64/dxgi.dll ~/.wine-sm64rt/drive_c/windows/system32/

# Set DLL overrides
WINEPREFIX=~/.wine-sm64rt wine reg add "HKCU\Software\Wine\DllOverrides" /v dxgi /d native,builtin /f

# Install VC++ runtime
WINEPREFIX=~/.wine-sm64rt winetricks -q vcrun2019 vcrun2022 d3dcompiler_47

Step 7: Run

sm64rt (vanilla textures + RT)

#!/usr/bin/env bash
export WINEPREFIX="$HOME/.wine-sm64rt"
export VKD3D_CONFIG=dxr
export WINEDLLOVERRIDES="dxgi=n,b"
export DXVK_HUD=fps
cd ~/build/sm64rt/build/us_pc
exec wine sm64.us.f3dex2e.exe

Render96ex RT64 (HD textures + HD models + RT)

#!/usr/bin/env bash
export WINEPREFIX="$HOME/.wine-sm64rt"
export VKD3D_CONFIG=dxr
export WINEDLLOVERRIDES="dxgi=n,b"
export DXVK_HUD=fps
cd ~/build/render96-rt/Render96ex/build/us_pc
exec wine sm64.us.f3dex2e.exe

8BitDo Controller Support

Add these environment variables to your launch script:

export SDL_GAMECONTROLLERCONFIG_FILE="$HOME/build/sm64rt/gamepad/gamecontrollerdb.txt"
export SDL_JOYSTICK_HIDAPI=1
export SDL_JOYSTICK_HIDAPI_8BITDO=1
export SDL_JOYSTICK_RAWINPUT=0

Download the community gamecontrollerdb:

curl -sL -o ~/build/sm64rt/gamepad/gamecontrollerdb.txt \
  "https://raw.githubusercontent.com/mdqinc/SDL_GameControllerDB/master/gamecontrollerdb.txt"

If your 8BitDo Ultimate Wireless (2dc8:3109) Y-axis doesn't work, add this line to the db file:

03000000c82d00000931000000000000,8BitDo Ultimate Wireless 2.4G,a:b0,b:b1,back:b10,dpdown:h0.4,dpleft:h0.8,dpright:h0.2,dpup:h0.1,guide:b12,leftshoulder:b6,leftstick:b13,lefttrigger:a5,leftx:a0,lefty:a1,rightshoulder:b7,rightstick:b14,righttrigger:a4,rightx:a2,righty:a3,start:b11,x:b3,y:b4,platform:Windows,

Important: Unplug and replug the USB dongle before each launch if the controller doesn't respond.

RT64 Config Options

Edit ~/.wine-sm64rt/drive_c/users/<you>/AppData/Roaming/sm64ex/sm64config.txt:

rt64_gi true              # Global Illumination
rt64_denoiser true        # Denoiser (reduces noise)
rt64_sphere_lights true   # Dynamic lights on objects
rt64_max_lights 12        # Simultaneous light sources
rt64_target_fps 60        # FPS target
rt64_upscaler 0           # 0=off, 2=FSR (DLSS doesn't work via Wine)

Known Limitations

DLSS doesn't work — DXVK masks the NVIDIA GPU vendor for XeSS compatibility, which breaks DLSS initialization. Use FSR (rt64_upscaler 2) or native resolution instead.
Controller quirks — 8BitDo controllers in proprietary mode (VID 2dc8) only expose hidraw on Linux, not evdev. SDL2 HIDAPI handles this but Wine's HID translation can lose the Y-axis. The gamecontrollerdb mapping fixes this.
First launch is slow — vkd3d-proton compiles DXR shader pipelines on first run (30-60 seconds). Subsequent launches use the cached pipelines.

Credits

DarioSamo — RT64 renderer and sm64rt
Render96 team — HD textures and models
vkd3d-proton team — D3D12 to Vulkan translation
danielbanariba — Linux cross-compilation, vkd3d-proton bug discovery and fix, guide

Name		Name	Last commit message	Last commit date
configs		configs
README.md		README.md
install.sh		install.sh
screenshot-render96-rt-castle.png		screenshot-render96-rt-castle.png
screenshot-render96-rt.png		screenshot-render96-rt.png
screenshot-sm64rt.png		screenshot-sm64rt.png

Navigation Menu

Repository files navigation

SM64 RT64 Path Tracing on Linux

⚡ Quick Install (One Command, Multi-Distro)

Manual Installation (if you want to understand each step)

Requirements

The Problem We Solved

Step 1: Install Dependencies

Step 2: Download SDL2 MinGW Development Libraries

Step 3: Build sm64rt

Apply cross-compilation fixes

Compile

Step 4: Build Render96ex RT64 (optional, HD textures + models)

Step 5: Build vkd3d-proton from master

Step 6: Setup Wine Prefix

Step 7: Run

sm64rt (vanilla textures + RT)

Render96ex RT64 (HD textures + HD models + RT)

8BitDo Controller Support

RT64 Config Options

Known Limitations

Related Links

Credits

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

SM64 RT64 Path Tracing on Linux

⚡ Quick Install (One Command, Multi-Distro)

Manual Installation (if you want to understand each step)

Requirements

The Problem We Solved

Step 1: Install Dependencies

Step 2: Download SDL2 MinGW Development Libraries

Step 3: Build sm64rt

Apply cross-compilation fixes

Compile

Step 4: Build Render96ex RT64 (optional, HD textures + models)

Step 5: Build vkd3d-proton from master

Step 6: Setup Wine Prefix

Step 7: Run

sm64rt (vanilla textures + RT)

Render96ex RT64 (HD textures + HD models + RT)

8BitDo Controller Support

RT64 Config Options

Known Limitations

Related Links

Credits

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages