alt Hacker NewsWe're not just stochastic parrots though, we can parrot things stochastically when that has utility, but we can also be original. The first time that work was done, it was sone by a person, autonomously. Current LLMs couldnt have done it the first time
They are much more than stochastic parrots.
I have never understood the stochastic parrot interpretation. LLMs (and general deep learning models) are not statistical/stochastic based models. Statistics trivially apply, as they apply to all measurements of judge-able behavior. But the models do not perform statistical operations, nor do their architectures form tunable statistically driven systems.
They learn topological representations of relationships. Entirely different from statistics/stochastics.
--
Within their "style" of cognition, LLMs are very creative. They readily propose solutions to problems involving uncommon or unique combinations of disparate topics.
Coming up with artificial examples is easy (and they come up naturally for me all the time).
I think the best characterization of LLM knowledge, reasoning and creativity is: extremely wide (in ability to weave topics and communication constraints - one shot), but somewhat shallow (not being able to reason too deep.)
Within those bounds, they far far exceed human capabilities.