In my mind the abstraction should allow for RPCs and being on the same machine should allow to optimise things a bit, this way you simply build for the general case and lose little to no performance.

Think of the loopback, my programs don't know (or at least shouldn't know) that IPs like 127.0.0.5 are special, but then the kernel knows that messages there are not going to go on any wire and handles that differently.