logoalt Hacker News

Eisenstein10/11/20241 replyview on HN

> The model is not interpreting anything. All that is being done is a multdimentional map lookup with statistics.

So what? Can you propose another method to make a computing device understand language? The method of the creation of the output does not stipulate anything about the nature of the thing creating it. If someone could map out a human brain and tell you how thoughts are made and added a 'all that is being done is' in front of it, does that make your thought creation trivial?

> An LLM on top of an LLM (i.e using context to generate inputs to an LLM) is just a fancier LLM.

This is called a tautology. You have not given any compelling reasons why an LLM cannot do anything, so calling something another LLM is not compelling either.

> To really understand all of this, all you need to do is look at how Transformer works, namely the attention block. There is no such thing as Query, Key, and Value in the sense of how they are implied to be used. The may as well be called A,B,C, as they are all learned in training, and can be freely interchanged in naming. All you do for inference is multiply the output vector by A,B,C to get 3 matrices, then multiply them together (technically with a scaling factor for 2 of them, but again, doesn't matter for which 2, and the scaling factor can be built into the matrix itself)

Here is how it works, so therefore it must meet some criteria I have imposed arbitrarily.

> So we have to rely on hacks like temperature to make it appear like reasoning, when its really not even close.

You still haven't produced any valid argument at all, for why one thing would be evidence of the other.


Replies

ActorNightly10/12/2024

A good example of how to type a comment and yet not say anything.

It should be pretty clear to anyone that human brains aren't just one giant compute functions with a limited set of outputs. There is no concept in your or my brain what 12074389762193867*2398720876324 is, but we can certainly figure it out, some even with good memory with complete sensory depravation.

If you disagree with this, you are entitled to your opinion, but your comments on the state of AI are just irrelevant.

show 2 replies