Isn't that kinda to be expected if you want to provide dynamic loading functionality (dlopen)?
Is the windows situation really all that different/better (with GetProcAddress in kernel32.dll)?
Not my area, but isn't it really only because glibc doesn't maintain stable interfaces across versions? If it did, you absolutely could use the same ld.so with different glibc versions. But it doesn't, so here we are.
The Windows situation is way different: every process is supposed to link against kernel32.dll, that's the public interface to the kernel. In Linux, glibc is just one of many C stdlib implementations, you can have many versions of glibc on the same system, etc.
In theory, ld.so could provide a stable interface to its dynamic loading capabilities independently of glibc. Then glibc would not have to be updated in concert with ld.so. There's no inherent reason that glibc should be the library shipping a dynamic linker - we could easily be in a world where libld was developed independently from libc.
On Windows, things are less modular, so it's less of an issue. That said, there's also weird shenanigans when it comes to the CRT (which can be statically linked) vs ntdll (which provides the actual linker implementation), that can make certain niche features of the linker misbehave (delay loading in particular is weird).