Proving and improving the Platonic Representation Hypothesis with JEPAs

I guess the fundamental question would be, for what domains can we pretrain with “good views”?

Part of the magic of next token prediction is it works to extract intelligence from the unstructured web corpus.

Can JEPA formulate a similar scale training set of “good views”?

I'm not sure how I feel about the Platonic Representation Hypothesis.

In some ways it seems like it should be profound, but part of me thinks it might just be saying 'There is an optimal way to perform dimensionality reduction and as things get bigger it is easier to appoximate'

> 33M-parameter text JEPA reaches .125 on the PRH paper’s benchmark, against .034 on Flickr captions, level with Qwen2.5-3B and -7B. They need fewer samples.

Im bullish on jepa models. Success of autoregressive next token prediction feels like a stroke of luck.

how come all the examples in demo have wrong answers

https://huggingface.co/spaces/galilai-group/prh_demo

> A man in a black shirt and hat with a yellow neckerchief has his hand raised.

no hand raised

> A white sheep sitting on top of a cake.

no its just cake sheep . sheep is not sitting

> A man is standing on the beach while holding his surfboard.

Its not just a man

infact all the examples are wrong.

Do memes have a concept of proto memes? Like primes are the proto numbers that all other numbers are made from? Could there be similar ontological primes for all concepts / semantic space? Base reality concepts that are irreducible so to speak.

I thought the platonic representation hypothesis was much more ambitious: wasn't it supposed to apply for any model across modalities (e.g. including human minds), rather than just the same model but with different modalities? I'll have to actually read the paper but I suspect there to be very strong assumptions which defeat the interesting aspects of the platonic representation hypothesis.

I built a ppt with claude for investors with the exact same phrase "AI infrastructure for proving and improving ... " on Oct 27 and sent it to investors. I suggested the title during the chat and claude agreed a lot. It ends up as a title of this paper 2 days later.