logoalt Hacker News

0x_rstoday at 12:51 PM0 repliesview on HN

Some projects or tasks might become impossible to do any debugging or work on in the future, because every bug is potentially exploitable with security implications or can be twisted into something against guidelines. And they're so popular, and any bugs in them so sought for, there's a massive negative signal associated with them. LLM cannot truly infer intent from the user, an innocent request is indistinguishable from a carefully crafted scenario from bad actors, so I would never trust anyone claiming those ambiguities can be solved in their product.

If some LLMs become too strict, they'll simply be impossible to reliably use, and hopefully fail along with their providers. Claude (only reasoning models, after 4) has repeatedly refused to perform translations for text that was not lyrics (poems), it's very stupid.