On Binary Translation and Its Consequences

[Disclaimer: I wrote Rosetta 2, so everything I say is biased by that.]

Contemporary out-of-order CPUs are incredibly over-provisioned; microarchitects will justify a new feature by another 0.1% gain on some benchmark. The end result is a CPU that's pretty decent at handling the sort of code bloat that comes from a binary translator.

There's also a big tradeoff in adding more optimizations to a binary translator. You would like to be able to precisely handle exceptions (especially ones caused by invalid memory accesses) while presenting a userspace exception handler with an architecturally valid state for the source program. There are some optimizations that would be easy to do in principle but are painful for maintaining this mapping between source program states and translated program states. The complexity burden combined with the difficulty of debugging that added complexity (or exhaustively verifying it up-front) shapes many of your decisions when writing a production binary translator. You should always have more Cool (tm) ideas than you actually use in practice.

When I bought my M1 Macbook, I had a firsthand experience with how well binary translation worked. It was almost perfect with one MAJOR exception - anything that used JIT - so stuff like Electron apps or Java stuff (IntelliJ-based IDEs). Which was kinda ironic - JITs were designed around the idea of portability across CPU architectures, yet usually they are one of the hardest pieces of code to port, and they make this sort of binary translation approach - which has relatively long compile times but good exec time - very slow and painful.

> Consumers in the PC segment expect high performance across a wide range of applications

It seems there is a persistent rift between what hardware designers think users want and what users are asking for, only recently being addressed by major OEMs with products such as the MacBook Neo.

Nobody I personally know wants a faster CPU; they want competent, faster, native feeling software and more power efficient hardware. CPUs have been plenty fast for at least a decade now, for anything the majority of users and even gamers are doing. I'm playing through the new and notoriously demanding LEGO Batman game on an RTX 2080 and an i7-4790k, a CPU from 2013 that even the base M1 easily beats. The game runs fine. Most people I am confident do not need better than the M1.