logoalt Hacker News

xpeyesterday at 5:42 PM0 repliesview on HN

> The stochastic parrot LLM is driven by nothing but eagerness to please. Fix that, and the parrot falls off its perch.

I see some problems with the above comment. First, using the phrase “stochastic parrot” in a dismissive way reflects a misunderstanding of the original paper [1]. The authors themselves do not weaponize the phrase; the paper was about deployment risks, not capability ceilings. I encourage everyone people who use the phase to go re-read the paper and make sure they can articulate what the paper claims and be able to distinguish that from their usage.

Second, what does the comment mean by “fix that, and the parrot falls off the perch.”? I don't know. I think it would need to be reframed in a concrete direction if we want to discuss it productively. If the commenter can make a claim or prediction of the "If-Then" form, then we'd have some basis for discussion.

Third, regarding "eagerness to please"... this comes from fine-tuning. Even without it (RLHP or similar) LLMs have significant prediction capabilities from pretraining (the base model).

All in all, I can't tell if the comment is making a claim I can't parse and/or one I disagree with.

[1]: "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" (Bender et al., 2021)