alt Hacker NewsI'm trying to keep an open mind and understand what the author is trying to say because he is credentialed.
His main point is that discoveries involve
1. Variation,
2. Evaluation, and
3. Selective retention.
He makes a jump saying AI is only capable of 1) and humans are capable of 1) 2) and 3). I don't know what makes humans special enough that they can do 2) and 3)?
In fact, the more you think of this it is kind of strange - in science humans can only do "evaluation" because they have access to the real world. They can evaluate a new drug because they can do it on people so it is not some inherent limitation of AI but rather access to physical realm.
Finally I want to ask a specific thing: how do you mathematically falsify what this person is saying? How can you formally prove that - no AI can not "evaluate"? I ask because I make AI evaluate a lot of people's claims and it works for me.
Replies
He actually says the areas in which AI has had the novel successes are those which can be evaluated (like coding or Go). Not that it can’t happen at all.
He's saying that pre-training an LLM alone can't do it, but if you run an LLM in a loop with tools (like any coding agent) then it can. Also, the technique his group came up with should be used more:
> This is the weakness of deep learning that is alleviated with a new algorithm that my group presented in Nature a couple of years ago. Our “continual backpropagation” made one small change: every so often a less-used neuron would be re-initialized to small random weights. This allows the variation to continue and plasticity to be retained.
Here's the paper: https://www.nature.com/articles/s41586-024-07711-7
It has a fair number of citations, but I haven't looked into how much it's used.