Firefox, VA-API and NVIDIA on Fedora 37

Image comes from https://www.nvidia.com/en-us/about-nvidia/legal-info/logo-brand-usage/

Some time ago I got borrowed NVIDIA GeForce GTX 1070 from my employer (Red Hat) and I finally managed to put it to a workstation instead of my own AMD RX 6600 XT.

I installed proprietary drivers from rpmfusion and to my surprise everything worked smoothly (except Atom on XWayland). Both Wayland and X11 Gnome sessions popped up, Firefox picked up HW accelerated backend (WebRender) with DMABuf support so it’s time to check VA-API.

Thanks to nvidia-vaapi-driver by Stephen “elFarto” Firefox may directly decode video on NVIDIA hardware. The driver translates VA-API calls from Firefox to VPDAU used by NVIDIA. I think you also need a decently fresh NVIDIA drivers which supports DMABuf (which is used to transfer decoded images between Firefox processes and render them as GL textures).

I hit three bumps on the road. First one is Firefox RDD sandbox. Firefox runs media decode in extra process (RDD) which restricts where decoder can access. It was adjusted by Mozilla folks for VA-API decode on Intel/AMD but NVIDIA needs some extra tweaks. Right now you need to disable the sanbox by MOZ_DISABLE_RDD_SANDBOX=1 env variable.

Next one is a bug in recent NVIDIA 525 driver series (which I got from rpmfusion) and I needed to use direct mode (whatever it is).

Last issue may be in Firefox itself. Broken graphics hardware may freeze whole browser on start or spread coredumps on every start. That’s being worked on as Bug 1813500 and Bug 1787182.

There’s a complete how-to for Firefox/NVIDIA/Fedora 37 available on Fedora wiki.

Nvidia-vaapi-driver playback performance is similar to what I see on AMD/Intel. It also correctly handles decoding of intermediate frames which is recent AMD NAVI2 bug (Bug 1772028, Bug 1802844) so the playback is smooth and I haven’t seen any glitches or CPU usage peaks.

There are few options for NVIDIA users how to run it.

If you have a workstation (or laptop) with one NVIDIA graphics card, it’s quite simple. On Fedora 37 you boot on noveau drivers and then install NVIDIA drivers from rpmfusion. With the proprietary drivers both Wayland and X11 Gnome sessions work fine (anyone to test KDE?) and hardware acceleration is enabled.

A bit different scenario comes with integrated Intel device and secondary NVIDIA one. I don’t see any reason why to use NVIDIA as Intel works pretty well with Wayland, VA-API and X11/EGL. But if you really want to set NVIDIA as primary, X11 may be better for you. Wayland on secondary NVIDIA GPU is supported by Sway Wayland compositor only which is a bit geeky (or I’m just too lazy to learn new shortcuts and get used to a new environment).

Anyway, if you need to use NVIDIA as your primary GPU, there’s a hope for you (and it’s not due to me). Give it a try and report eventual bugs at Mozilla NVIDIA VA-API bug tracker.

Firefox with VA-API for brave Fedorans

A rocket propelled butterfly? – https://bugzilla.mozilla.org/show_bug.cgi?id=1772028

It’s been a long journey since the first VA-API implementation in Firefox. Two years ago Firefox 77.0 come to Fedora with accelerated video playback on Wayland which was more a tech preview than a working solution.

Since then X11 support was added, fixed many bugs, AV1 decoding was implemented so we can claim VA-API code as mature enough to enable it for testing in Firefox Nightly 103. As we don’t want to scare peaceful Ubuntu users and grandmas watching their favorite show, VA-API is enabled in Nightly channel only and stock Firefox 103 won’t be shipped with that.

But ‘Real Men‘ wants more challenge. ‘Quiche Eaters’ can use polished software, LTS distros or even Mac. That’s nothing for adventurers running on the edge. Thus new Firefox updates (Fedora 35, Fedora 36) has VA-API enabled by default ahead of upstream to get what you asked for, brave Fedorans.

Along with a bunch of upstream VA-API backports I also added a fix for mozbz#1735929 to make life easier for those who did a life mistake with NVIDIA hardware. I hope you’ve got a lesson and your next card will be AMD.

And how to fruitify it? It’s embarrassing it doesn’t need any complicated, cryptic, powerful, unforgiving and dangerous edits of Firefox about:config preferences, setting env variables on terminal or midnight vows. If VA-API is configured property Firefox just takes it and claims at about:support page.

VA-API is enabled

More details are available at https://fedoraproject.org/wiki/Firefox_Hardware_acceleration

Stand With Ukraine

Firefox 94 comes with EGL on X11

(In)Famous WebGL Aquarium demo. EGL brings you more fish 🙂

Firefox 94 is coming out next week and brings awesome news. OpenGL EGL backend its enabled by default on Intel/AMD and recent Mesa for users on X11.

This project has been driven by Robert Mader (most of the EGL work), Andrew Osmond (glxtest fixes and config), Jamie Nicol (EGL/Android and partial damage), Greg V (partial damage support) and Jan Ikenmeyer (Darkspirit) (help with issues, testing).

Historically Linux comes with GLX (OpenGL X11 extension) but that era is finally ending and we’re moving forward to EGL which promises all the goodness you can expect from modern graphics subsystem…or you can create a texture over graphics memory at least 🙂

I’ll keep aside all EGL / GLX difference and focus to changes from user perspective. GLX is old, well debugged and tied closely to X11 which means seamless experience and wide support by gfx drivers (like proprietary NVIDIA ones). It’s used by most X11 applications and ‘just works’.

EGL is ‘new’ from Linux desktop perspective and used mainly by Wayland, Android and various small devices. It’s not fully supported by all desktop drivers and has glitches (broken rendering of transparent windows for instance). But as Wayland is gaining momentum also EGL is getting more attention and fixes on Linux desktop.

And why we actually want EGL? Because it gives us a cool toy – EGLImages (and EGLFence). EGLImage is an object which is created over a piece of GPU memory (which can be DMA-Buf), shared with different process, used as a frame buffer (target of GL rendering) or a texture (source of GL rendering).

EGLImage allows to use GPU memory in a very creative way. VA-API decoded video frames or WebGL scenes can be mapped as EGLImages, moved from decode process to rendering process and used as a texture. EGLFence allows to lock EGLImages across process so we don’t re-paint WebGL scene while it’s used in a different process or recycle VA-API video frames too early.

And what you can expect from EGL in Firefox? Faster WebGL rendering (used on Google Maps for instance), more effective rendering due to to partial damage support and potential VA-API video decoding (that’s blocked by Bug 1698778 on both Wayland and X11). It also unifies rendering path for Wayland and X11, which means X11 will gain features done for Wayland (suspended rendering for invisible windows, better VSync support and more).

NVIDIA is also working on EGL & DMA-Buf support in their proprietary drivers so there’s a hope for owners of such FOSS unfriendly hardware.

So give Firefox 94 a try. If anything goes wrong, please file a bug. You can also disable EGL and switch back to GLX. Go to about:config page and flip gfx.x11-egl.force-disabled preference and restart browser.

Firefox Wayland development in 2021

I swear, no more crashes on Wayland!

It’s been long time from my last update about Firefox news on Linux and I’ve finally got some time to sum up what we’ve been working on for last year and what’s coming. There haven’t been introduced any new exciting features (from Linux perspective) for the last year but rather a hidden but important changes.

From Linux desktop developers perspective 2021 is a year of Wayland. KDE has been shipping decent Wayland compositor which becomes default for Fedora 34. It’s actually pretty fast and gives you smooth feeling of “good old times” with X11/Gtk2/name-your-favorite environment where any graphics change was just instant without lags or slow transitions. I must mention Robert Mader who created a new Firefox Wayland SW backend for the KDE.

As major desktop distro (Ubuntu) is slowly moving to Wayland we’re getting more and more Wayland/Firefox users. Even notorious troublemaker (NVIDIA) decided to step in and support it.

What’s done for next releases?

It’s good that Wayland market share is rising but we also need to make sure that Firefox is ready to run there without any major issues and matches its X11 variant. There are two major areas where Firefox is behind its X11 counterpart – clipboard and popup handling. It’s given by some Wayland protocol features where we can’t simply duplicate the X11 code here.

Clipboard on Wayland is similar to X11 one but we need to translate Wayland asynchronous clipboard to Firefox/Web synced one. I tried various approaches but the best one seems to be just use the asynchronous Wayland clipboard as is and implement some kind of abstraction over it. That was implemented in Firefox 93 an it’s going to be shipped by default in Firefox 94.

On the other hand popups are the most annoying thing we have to implement on Wayland. Firefox just expect any popup can be created any time without its parent (or use main window as a parent) but Wayland requests strict popup hierarchy. It means every window can have only one child popup. When more than one popup is opened it has to be attached to the previously opened popup which becomes a parent for it. And when any popup in the chain is closed the popups must be rearranged to keep the chain connected. This involves all kind of popups like content menus, tooltips, extensions popups, permissions popups and so on. Plus there are some interesting bugs in Wayland protocol or Gtk so excitement/frustration is guaranteed and basic popup implementation becomes extraordinary challenge where small changes can introduce various regressions. Despite the ‘fun with popups’ the popup tracker is almost clear and we’ll ship it in Firefox 94.

One of main Wayland feature is support of monitors with various DPI/scale factor together. Fedora default compositor Mutter shows here a creative approach and reports screen sizes differently than other compositors. As we really want to know screen sizes Firefox tracks monitor changes from Wayland directly and find correct screen by matching screen left top corner point – which fortunately stays stays same for all compositor. We also stop painting Firefox window when screen scale changes so you should enjoy seamless experience on systems with various screen sizes with Firefox 93.

Future plans for Firefox 95

Firefox 95 development cycle begins next ween and I’m going to look to drag and drop which has been partially broken for long long time. Some Wayland specific fixes are already in Firefox 94 but we need to rework some parts to correctly copy files from remote destinations (like inbox) to local filesystems, fix names of dropped files or do tab preview of moved tabs. There are also new interesting compositors bugs as usually 🙂

Future plans for Firefox 96

Firefox Wayland port is generally done and there isn’t any big difference between X11 and Wayland variant at least on GNOME which Fedora uses as default environment. We’re fixing minor bugs and keep eye on user reports.

For next quarter I’d like to look at GPU process for Wayland. GPU process is running tasks related to graphics hardware and shields browser from HW drivers crashes. It’s also place where VAAPI video decoding should be run and will be properly sandboxed there (right now VAAPI is run in content process along general Firefox code, it’s restricted by content sanbox which leads to various VAAPI code crashes and failures).