Access pattern is mitigated by texture swizzling which will happen regardless of how it's ultimately rendered. So even if drawn with an identity matrix you're still "paying" for it regardless just due to the underlying texture layout. GPUs can sample from linear textures, but often it comes with a significant performance penalty unless you stay on a specific, and undefined, path.