logoalt Hacker News

SpicyLemonZestyesterday at 6:04 PM1 replyview on HN

What is an example you have in mind of a self-fulfilling prophecy? I genuinely don't know what you could be referring to. It seems to me that they keep making surprising prophecies, and the popular reaction to them seamlessly transitions from "that's crazy, no way it will happen" to "that's silly, it's just a cloud pattern". Did you find it obvious or self-fulfilling in 2025 that LLMs would soon be able to resolve open questions in mathematical research?


Replies

orbital-decayyesterday at 7:05 PM

Self-fulfilling prophecies are social effects, not real predictions. Anthropic's "We predict our model misbehavior potentially being able do destroy the world in 1% of the cases" -> "We want to find the evidence of our model misbehaving, and we want it bad" -> "See, our model is hacking our rewards and has functional emotions, this means it's misbehaving with the intention of destroying the... HUMANITY!1" -> repeat x100, manipulate the media into amplifying it x10000 for clicks -> people are begging to safeguard them from the evil AI. Which is already likely to happen, the average layman's Overton window already includes the fantasy of rogue AI.

None of that was real or remotely dangerous in the first place, of course. It wouldn't have resulted in controls, had they not been scaremongering. This will end in extreme fascism or people getting enslaved "for their own safety", and it won't even require malicious intent, only incentives, detachment from reality, and confirmation bias. Although it doesn't exclude malice either.