logoalt Hacker News

exceptione • last Saturday at 9:28 AM • 1 reply • view on HN

  > we do NOT understand how LLMs work AT all.
  > We Do Not understand Why or How an LLM produced a specific response for a
  > specific prompt.
You mean the system is not deterministic? How the system works should be quite clear. I think the uncertainty is more about the premise if billions of tokens and their weights relative to each other is enough to reach intelligence. These debates are older than LLM's. In 'old' AI we were looking at (limited) autonomous agents that had the capability to participate in an environment and exchange knowledge about the world with each other. The next step for LLM's would be to update their own weights. That would be too costly in terms of money and time yet. What we do know is that for something to be seen as intelligent it cannot live in a jar. I consider the current crop as shared 8-bit computers, while each of us need one with terabytes of RAM.

Replies

ninetyninenine • last Saturday at 3:03 PM

https://www.youtube.com/watch?v=qrvK_KuIeJk&t=284s

For context, George Hinton is basically the Father of AI. He's responsible for the current resurgence of machine learning and utilizing GPUs for ML.

The video puts it plainly. You can get pedantic and try to build scaffolding around your old opinion in attempt to fit it into a different paradigm but that's just self justification and an attempt to avoid realizing or admitting that you held a strong belief that was utterly incorrect. The overall point is:

   We have never understood how LLMs work.
That's really all that needs to be said here.