It's sort of explained in the article, actually.

Upscaling and related applications are already supported by cooperate matrix ops. Those shaders/kernels run in workgroups that operate on a tile of pixels at a time, so you naturally get matrix-matrix operations. And yes, at least in Vulkan, there has been cross-vendor support for those for a while now.

Neural materials and similar applications often operate on a single datum at a time, like in a pixel shader or a raytracing shader.

Yes, the hardware still groups the threads of a pixel shader together, but there is inherently more divergence. And so the natural program representation tends to have matrix-vector operations instead of matrix-matrix operations.

That's where cooperative vectors come in.