There must be some quirk of whatever combination of software packages are installed there, but the proprietary drivers are not the culprit, at least not alone (i.e. there could be some interaction with other software packages with which I have little experience, like Gnome).
I have been using the proprietary NVIDIA drivers for more than 2 decades on various hardware, both desktops and laptops, mostly with Gentoo Linux.
Opening an OpenGL context or any other OpenGL operations have always been instant.
So the usual self inflicted misconfiguration then.
In similar style I recently wiped a device that I thought had firmware that was slow to boot but it turns out that a hang and subsequent timeout due to something I had long ago misconfigured had been obscured by the previous setup that defaulted to hiding all details during boot.