Except of course with Nvidia drivers, those are always shit.

Doesn't that depend on the distro? In most cases they should be supplied as a (meta)package and only require installation through the package manager, kernel modules should be built automatically then.

While this is ofc only anecdotal evidence: I haven't had problems with different models of Nvidia GPUs on different distributions (OpenSUSE, Debian, Pop!_OS, Elementary, EndeavourOS) in the last years. With a small workaround, even Wayland works flawlessly - the problem with missing GAMMA_LUT support and night light notwithstanding here.