They did test for correct interpretation, so what point are you making?

>I can interpret weights too, but will that interpretation be correct?

In TFA case, given that they were able to guess what the weights did (which the researchers knew in advance and could cross-check) and to even tweak and improve them, the answer is yes.

Biblically speaking, spirits from the vasty deep must be demons since they are from below and angels, God and Jesus are from above. So in that sense it’s not very impressive. Demons are something a sane person would rather avoid.

> I can interpret weights too, but will that interpretation be correct?

Chances are much higher than a model can figure out what the patterns are, decipher them, and retain that knowledge in a meaningful way it can reason about. Me and you might take years or months to do all of that on our own, with a "map" of the weight we might be able to discuss it, but not as effortlessly as an LLM.

Honestly, I had told a few different colleagues that reading a model is probably not that exclusive to frontier models. It's probably yet another reason OpenAI / Anthropic don't release any big open weight models, but thankfully Google and Microsoft release reasonably sized models. Imagine a new GPT open weight release, it would get dissected drastically.

Further than that, aren't "weights" just another sequence of tokens? We see them as vectors, pass to the API or as strings into a chatbox interface. Is it so surprising that the high-dimensionality linear algebra machine crunches them and gives us back useful vectors that can be mapped to text?

Isn't that what this whole technology paradigm is about? How is it surprising?

Granted, it is cool. But I see so much of "look at this cool thing I did with LLMs" and honestly... we need more structured text, more explicit documentation, more ergonomic and intuitive CRUD applications.

The amount of "look at cool things" is growing way faster than before and they are all sort of low effort now because the "agentic" LLM-VM looping apparatus can do a lot of heavy lifting and run with our prompts.

And I mean, if one has jargon, knowledge, and intention, they can also leverage that into great structured text, ergonomic and intuitive CRUD apps etc, all of which add great value to our mundane pursuits of applied research, software products and services and so on.

Honestly, I begin to think the problem is people dazzling themselves instead of doing the housekeeping that is the bulk of life. Technology choice is entirely secondary.

And this reminds me of the king in the little prince, who had the power to order the sun to rise or go down. But only once a day at a certain time.

But the whole point of the article was that it was correct.

Why do this without testing? Why would you assume that?