LLMs also try and find short cuts to get the task done, for example I wrote some code (typescript) for work that had a lot of lint errors (I created a pretty strict rule set)

And I asked codex to fix them for me, first attempt was to add comments to disable the rules for the whole file and just mark everything as any.

Second attempt was to disable the rules in the eslint config.

It does the same with tests it will happily create a work around to avoid the issue rather than fix the issue.