alt Hacker NewsThe focus on user space fragmentation is wrong, IMHO.
One of the maintainers (Catalin Marinas) made [0] a much more important point: Apple makes no promises about how their "TSO" bits work now or will work in the future. This mode was designed for Rosetta2, not the general public. It is not documented formally. Someone saying "it is TSO" is not documentation. A formal definition of a memory model is usually a very long document describing a lot of corner cases, for example [1] is a SUMMARY of the ARMv8 memory model, it is 31 pages long. It is a summary! The full spec makes up chapters D7 and D8 in [2], totaling 243 pages. Even there, there are corners that it does not touch on and people get wrong. Without such a spec for Apple's TSO mode, how can anyone rely on how it might or might not work?
Additionally, you might find silicon bugs if you do something in this mode that Rosetta2 doesn't or didn't. Consider that the only first-party user of this mode was Rosetta2. Anything it does not do that you do might find a bug.
The stated linux kernel policy of "do not break user space" is impossible to deliver on, if built on an undocumented hardware feature that might change at any time and was never fully publicly specified. The maintainers are right to reject this.
[0] https://lwn.net/ml/linux-kernel/[email protected]/
[1] https://developer.arm.com/-/media/Arm%20Developer%20Communit...
[2] https://documentation-service.arm.com/static/6943ef0c7982093...
Replies
It is actually documented now: https://developer.apple.com/documentation/virtualization/acc...
As mentioned in the article, TSO is not exclusive to Apple's ARM implementation.
> Some NVIDIA and Fujitsu CPUs run with TSO at all times; Apple's CPUs provide it as an optional feature that can be enabled at run time.
Linux never had "must be publicly documented" as a general rule.
I'd posit that Linux having a hard policy of "do not break user space" has always just been "mostly true", especially around architecture specific stuff, and stuff tied to corporations.
I bring this up only because recently, I made it my mission to get IBM's "PowerVM Lx86" dynamic translation system running on a POWER9 system running modern Debian Linux.
This required a lot of hackery I won't go into here, but it revealed two things to me:
1. The lx86 system depended on an undocumented (??) pseudo-syscall to switch between big and little endian modes. This was "syscall 0x1EBE", which was implemented in the exception handler for PowerPC. In other words, it wasn't a real syscall, and tools like strace do NOT capture/log it. It was a conditional branch in the assembly entry point for the process exception handler which switches endianness and then returns. Quicker than a "real" syscall. Also, long gone in the Linux kernel, replaced with syscall switch_endian, hex 0x16B. Adding this in wasn't too hard, but it'd sure as heck never make it upstream again ;)
2. A lot of other Linux calls have had bits added which break old applications that are "rigidly coded". For example, netlink has had extra data stuffed into various structures, which code like lx86's translator panics on. To get network stuff running, even with an antique x86 userland, required removing a bunch of stuff from the kernel. Not disabling it via configuration, but deleting code.
All this to say, there is a precedent for breaking the Linux user facing ABI for hardware features like this. I'm not saying that's a good thing, but it is a thing.