alt Hacker NewsI don't see how its output possibilities could be infinite given a finite set of data. Taken to the extreme if you give an LLM only a single piece of information then that is going to be the only thing it can produce. Given it two pieces of information and maybe it can produce every combination of those two things. Given all of the worlds knowledge I'm sure its possible outputs are incomprehensibly large, but still not infinite. Then having its outputs match a set of really specific instructions is going to be a small subset of that.
Replies
Sure, a given LLM can only produce so much (though it might be a lot), and then you'll need fresh inputs. But you contend that those have to be human, while I am arguing that they could be
- random,
- from the environment,
- from other AIs.
That increases the space considerably.
In particular, note that your argument also applies to humans: A human, being finite, can only produce finitely many things. But you add other humans, the environment, and pure chance to the mix, and it'll be pretty infinite for all practical purposes.
Well humans do that and you expect them to continue to do that - why can't AI models do it as well?
The set of data will increase as it outputs
I don't think AI will continue growing infinitely either, but maybe "incomprehensibly large" is large enough to solve all the math problems.
How is this different from human brains being made up of a finite number of neurons in a finite number of configurations? I know very little about LLMs and nothing about neurology and maybe you can explain to me why one is so much more powerful than the other.