Some time ago I got borrowed NVIDIA GeForce GTX 1070 from my employer (Red Hat) and I finally managed to put it to a workstation instead of my own AMD RX 6600 XT.
I installed proprietary drivers from rpmfusion and to my surprise everything worked smoothly (except Atom on XWayland). Both Wayland and X11 Gnome sessions popped up, Firefox picked up HW accelerated backend (WebRender) with DMABuf support so it’s time to check VA-API.
Thanks to nvidia-vaapi-driver by Stephen “elFarto” Firefox may directly decode video on NVIDIA hardware. The driver translates VA-API calls from Firefox to VPDAU used by NVIDIA. I think you also need a decently fresh NVIDIA drivers which supports DMABuf (which is used to transfer decoded images between Firefox processes and render them as GL textures).
I hit three bumps on the road. First one is Firefox RDD sandbox. Firefox runs media decode in extra process (RDD) which restricts where decoder can access. It was adjusted by Mozilla folks for VA-API decode on Intel/AMD but NVIDIA needs some extra tweaks. Right now you need to disable the sanbox by MOZ_DISABLE_RDD_SANDBOX=1 env variable.
Next one is a bug in recent NVIDIA 525 driver series (which I got from rpmfusion) and I needed to use direct mode (whatever it is).
Last issue may be in Firefox itself. Broken graphics hardware may freeze whole browser on start or spread coredumps on every start. That’s being worked on as Bug 1813500 and Bug 1787182.
There’s a complete how-to for Firefox/NVIDIA/Fedora 37 available on Fedora wiki.
Nvidia-vaapi-driver playback performance is similar to what I see on AMD/Intel. It also correctly handles decoding of intermediate frames which is recent AMD NAVI2 bug (Bug 1772028, Bug 1802844) so the playback is smooth and I haven’t seen any glitches or CPU usage peaks.
There are few options for NVIDIA users how to run it.
If you have a workstation (or laptop) with one NVIDIA graphics card, it’s quite simple. On Fedora 37 you boot on noveau drivers and then install NVIDIA drivers from rpmfusion. With the proprietary drivers both Wayland and X11 Gnome sessions work fine (anyone to test KDE?) and hardware acceleration is enabled.
A bit different scenario comes with integrated Intel device and secondary NVIDIA one. I don’t see any reason why to use NVIDIA as Intel works pretty well with Wayland, VA-API and X11/EGL. But if you really want to set NVIDIA as primary, X11 may be better for you. Wayland on secondary NVIDIA GPU is supported by Sway Wayland compositor only which is a bit geeky (or I’m just too lazy to learn new shortcuts and get used to a new environment).
Anyway, if you need to use NVIDIA as your primary GPU, there’s a hope for you (and it’s not due to me). Give it a try and report eventual bugs at Mozilla NVIDIA VA-API bug tracker.
Martin Stransky's Blog 