alt Hacker Newsafaik, you always have to break open abstractions for more performance. If you ignore cache-levels in your program you're gonna have a bad time - and depending on the system the layout (and with it how you should use it) is different. The same is true for how machines are interconnected. Depending on the wiring you have different throughput-values when sharing data between nodes. The whole area screams "not universal" to me.
I think people working on these abstractions would claim that an appropriate abstraction does involve the exact features you mention. Cache hierarchies and vectorization are common to CPUs and GPUs - in some sense, the numbers parametrizing them are all that differs. With a good abstraction, this machine-tuning can be automated.