That's... a good result, actually. No, I'm serious.
This reads exactly like my inner thought process on a novel or tricky task I'm asked to solve, especially when I know I'm tired (or drunk, back in the times I consumed alcohol on a regular basis), and need to spell everything out (out loud or in a text file).
Hell, it's exactly how I expect a kid who just learned about fractions would think. I have a vague recollection I processed such tasks this explicitly as a kid, until I understood the topic.
LLMs pulling this off reliably? That's huge progress. I used to say[0] that GPT-4 is best imagined as a 4 year old kid that memorized half the Internet. But this? This is 8 year old's stuff.
--
[0] - I currently prefer comparing it to "inner voice", and its performance and propensity to hallucinations to a smart schoolkid that's being asked questions by the teacher about things they only read about but didn't fully process, and who's pressured into giving some answer, as saying "I don't know" is an instant F and public humiliation. Such kid will be forced to extrapolate on the spot, but if they're smart enough and remember enough, they'll often get it at least partially right. I know that from personal experience :).
alt Hacker News
That's... a good result, actually. No, I'm serious.
This reads exactly like my inner thought process on a novel or tricky task I'm asked to solve, especially when I know I'm tired (or drunk, back in the times I consumed alcohol on a regular basis), and need to spell everything out (out loud or in a text file).
Hell, it's exactly how I expect a kid who just learned about fractions would think. I have a vague recollection I processed such tasks this explicitly as a kid, until I understood the topic.
LLMs pulling this off reliably? That's huge progress. I used to say[0] that GPT-4 is best imagined as a 4 year old kid that memorized half the Internet. But this? This is 8 year old's stuff.
--
[0] - I currently prefer comparing it to "inner voice", and its performance and propensity to hallucinations to a smart schoolkid that's being asked questions by the teacher about things they only read about but didn't fully process, and who's pressured into giving some answer, as saying "I don't know" is an instant F and public humiliation. Such kid will be forced to extrapolate on the spot, but if they're smart enough and remember enough, they'll often get it at least partially right. I know that from personal experience :).