alt Hacker NewsThe Platonic Representation Hypothesis initially feels profound but actually seems inevitable after some thought because all neural networks are trying to model the same reality. It would actually be weirder if it wasn't true and neural networks could form completely incompatible representations of the same reality. In some sense it seems similar to reducing different math equations to the same most simplified form
Replies
Isn’t big-endian vs little-endian a trivial example of how you can represent the same mathematical object (integers) in mutually-incompatible ways?
Maybe I’m missing the detail on how complex the mapping function can be before we say the representations are incompatible.
It might be weirder, but part of human intercommunication issues lies in the fact that people live with very different representation, though they can be supposed to live in the same single encompassing one. They don't all experience all of the world and even when they share some common experience they went through, they didn't experiment them from the same ground and previous experiences do influence current and future interpretations.
I find this fascinating because many philosophers in the wake of Wittgenstein and Kripke's work on rule following still believe the contrary. I've always thought they're wrong if you take into account constraints of economy (and shared perception, pace Quine), and that e.g. Solomonoff Induction already shows that but I seem to belong to a minority in that respect. It's interesting to see similar ideas of convergence to a unified representation show up in machine learning by neural networks.
I think it's might be possible that "neural networks could form completely incompatible representations of the same reality" except that it's humans who get to decide if a representation is good (i.e. compatible with the human perception of reality) or bad and NNs that differ are dismissed as erroneous.