alt Hacker NewsThere’s not such a straightforward relationship between safety and model sis.
According to the book The Thinking Game, lower quality models at that time were considered less safe, because they could be easily tricked into doing harmful stuff. In the book, Dario (of Anthropic) was the head of safety at openAI and was responsible for pushing for 10x scaling in training to make the models safer.
It does make sense, a smart model is going to be way harder to jailbreak into telling me how to synthesize anthrax (or something).
Replies
Flavius • today at 5:57 PM
> According to the book The Thinking Game...
According to me, this is nonsense.
➕ show 1 reply
Models are quite safe when they're useless, actually.
In the times of GPT-3 I'd scoff at the idea of an LLM doing any hacking; today, I'm running several AIs on my code before publishing, and they are finding (and demonstrating!) RCEs on my localhost server.
For example, one found a missing check in a third party JWT library which allowed full account takeover, which I'd have never even looked at.
Hence I don't believe a single word coming out of these people's mouths. Their "beliefs" are just marketing.