I generally agree with this opinion and would love to see a proper well documented low-level API for working with GPU. But it would probably result in different "GPU ISAs" for different vendors and maybe even for different GPU generations from one vendor. The bloated firmwares and drivers operating on a higher abstraction level allow to hide a lot of internal implementation details from end users.
In such world most of software would still probably use something like Vulkan/DX/WebGPU to abstract over such ISAs, like we use today Java/JavaScript/Python to "abstract" over CPU ISA. And we also likely to have an NVIDIA monopoly similar to x86.
I wouldn't be so sure, as if we analogize to x86(_64), the ISA is stable and used by many vendors, but the underlying microarchitecture and caching model, etc., are free reign for impl-specific work.
There’s a simple (but radical) solution that would force GPU vendors to settle on a common, stable ISA: forbid hardware vendors to distribute software. In practice, stop at the border hardware that comes from vendors who still distribute software.
That simple. Now to sell a GPU, the only way is to make an ISA so simple even third parties can make good drivers for it. And the first successful ISA will then force everyone else to implement the same ISA, so the same drivers will work for everyone.
Oh, one other thing that has to go away: patents must no longer apply to ISAs. That way, anyone who wants to make and sell x86, ARM, or whatever GPU ISA that emerges, legally can. No more discussion about which instruction set is open or not, they all just are.
Not that the US would ever want to submit Intel to such a brutal competition.