alt Hacker NewsYou seem to be missing their point (which I agree with). The type of intelligence we are equipped with allows us not to have the level of memory an LLM does and still complete tasks that are novel to us every single day. Like navigating a shopping cart through tricky coridors in a store, coming up with a dad joke as in sibling example, combining a set of tools to achieve something we have never seen before, etc.
LLMs approximate a lot of that very well by simply having seen it before.
Also watch kids develop language: they learn patterns with much less training data than LLMs.
I addressed much of this in my response to a sibling comment, but a few more here:
> novel to us every single day. Like navigating a shopping cart through tricky coridors in a store
We have been practicing navigating the physical world for something like 16hrs/day every day from the moment of our birth. All the sensory data passing through our brains during that time is far larger than any dataset an LLM is trained on.
Humans navigating a shopping cart at a store have likely navigated the physical world before, pushed a shopping cart before, and in combination have navigated stores while pushing shopping carts before. Nevertheless, many still bump into objects all along the way.
Them succeeding at successive variations of store layouts is not novel unless we expand the definition of novel to mean any recombination whatsoever of pre existing concepts.
I’m certain that with all the intense usage of AI by hundreds of millions of people, there have been countless collections of words passed to LLMs so far that have never before been uttered in exactly such a sequence, let alone in the dataset.
I’m equally certain the LLMs have responded to those words with collections of its own that have also never been uttered in that exact sequence, responding to their unique context.
It is trivial to produce an example of this now yourself if you’d like.
The LLM we’re talking about, mentioned in the OP, has never seen this solution to this problem in its dataset. A large number of brilliant mathematicians were not able to discover this solution. They are themselves expressing that this is a novel breakthrough and had this come from a human it would be treated as such.
If the response to that is “well it’s just recombining concepts it already knows until it finds a solution that works” I would ask how that differs from what humans do?