Yep, 100%. In fact writing anything GPU co-processor related is nothing like porting CPU code. Sometimes the entire kernel needs to be rewritten outright when switching between GPU gens (of the same vendor). Things like precision et al also vary massively.

Usually these architectural differences are handled by the intermediate platform layer, which often doesn't apply if you're writing low-level kernels. Although this

> I'm guessing (but not knowing) that small subtle differences in matrix multiply across different vendor's architectures (and product generations of an individual vendor's architecture) is responsible for a good portion of software crashes when trying to run a local LLM on a different architecture or with a different stack (ROCm vs. CUDA, for example) than the ones it has been explicitly tested on.

Has to do with everything else but matrix mult precision. Feature support/drivers/alignment generally causes crashes, and again, the GPU field situation is so precarious that it's 10x worse than the AVX512 segmentation in the CPU world. You basically need to microverify whether the target GPU supports a given instruction/feature.