Haven’t thought too deeply about this, but as a physicist we’d do things like switch from Euclidean to Spherical coordinates or a completely different dynamical representation like the Langrangian, and this makes certain problems dissolve (or at least become much simpler to solve analytically).

Does it even make sense to claim there is a “platonic” / true representation? Maybe there is an optimal one for a specific task, eg representing a small map is great in the Cartesian frame. But you need a spheroid instead of a plane to accurately model the earth, so the latter is “more real” in some sense. But these seem like engineering tradeoffs of error vs compute and a given implementation could correctly pick either depending on its constraints.

Edit: reading the paper more I think this is a “RTFA” point, their “PRH” is a very explicitly defined mathematical theorem and as such my hand-wavey thoughts above don’t really apply.

What if we ultimately find there are two different, internally consistent but mutually incompatible, representations?

What if one of them is a Republican and one is a Democrat? We already see this in some systems today.

More worrying, what if one is (for lack of a better word, what we might describe as) God and one is the Devil?

It’s less profound sounding if instead of talking about “representations of the world” you talk about compressed representations of data or decision boundaries in a classifier over data.

Seems to me that it matters because it has implications for how we can train smaller models?

The Platonic Representation Hypothesis initially feels profound but actually seems inevitable after some thought because all neural networks are trying to model the same reality. It would actually be weirder if it wasn't true and neural networks could form completely incompatible representations of the same reality. In some sense it seems similar to reducing different math equations to the same most simplified form