alt Hacker NewsAnecdote: I have seen it cheat / bend the rules with my own eyes.
In hindsight, this shouldn’t be surprising. The prompt requests to make a certain operation faster, the LLM will try to find a way and isn’t even informed on the wider context so it myopically chases the goal.
My own anecdote: I have seen frontier models get stuck repeating the word "the" until it fills up the context. I really don't care to argue the capabilities of LLMs or their somehow simultaneous lack thereof; I'll just reiterate that ultimately the operator chose to ship a cheat.