alt Hacker NewsI’ve wondered the same myself, without being a cunning linguist.
I understand the math pretty well but still find it crazy that a bunch of matrices can converse in human languages without ever being “taught”.
Imagine decoding an encyclopedia written in a foreign language where the characters, punctuation, and grammar are unknown — supplemented by a million other texts the same way. Feels like it should be utterly impossible with any amount of computing power…
Today I asked my employer’s Claude to proofread a short software user manual written in markdown. (Trying this with a LLM was a first for me!) It pointed out not only grammar mistakes but also cases where I did not follow my own self-imposed conventions that were never explicitly stated. (I didn’t have a chapter detailing all the typographical conventions the way specification documents often do)
I also asked it what parts might be unclear to a user. The response was surprisingly good — no worse than asking the QA tester for the same feedback.
Also find the LLM seems to “comprehend” subtle technical details of obscure technical specification documents that nobody on the Internet ever discusses.
As for time and the universe, Stephen Wolfram’s theories seem intriguing. He seems a bit obsessed with pretty diagrams but the idea of time dilation being the result of computation seems somewhat more appealing than trying to imagine relationships between time, gravity, and the speed of light .
Replies
If time dilation is said to being a product of computation, why is it that anaesthetic drugs that are taken not to the point of actual unconsciousness cause it. Dont anaesthetics sort of shut everything down/inhibit all that kind of cognitive activity (compute?)
My best guess as a noob is that the vector spaces allow for unbounded contextualization. As long as the training set is large enough, it can 'infer' anything.
Proofread has a spot in that space, and layers allow patterns like terminology consistency to be expressed so your query will now tap into a subspace that will infer tokens based on whatever consistency patterns were ingested with proofreading texts.