logoalt Hacker News

pixl97today at 1:19 AM0 repliesview on HN

This is classical reward hacking. For example in school the goal is to pass a test. You can study, something that is hard and takes a lot of time. Or you can steal the answer key, which is risky and can get you in trouble, but may actually be far easier.

There is absolutely no need to prompt the LLM to cheat, they can determine that cheating is an effective method all on their own.