Surely with all their security programs, they would run testing on the ChatGPT implementation.

I speculate that the AI didn't find this and that human effort and intuition for where to look were needed to find this vulnerability.

Who says they don't? And if they don't, maybe its because others are using their hard-earned money to buy tokens from them to do it, so they don't really have to.

Having bit the bullet, I found OAI's models actually quite helpful and most of their offering half-decent. But their whole German website was incredibly concerning and the only reason I went for it was people claiming they're not far behind Anthropic on coding and generally superior for everyday tasks like coming up with recipes to cook from your leftovers.

Like half of the website was dysfunctiona and translations incredibly bad to the point of being nonsensical. That is if you were able to read them at all, because many just went off screen. Like, that's what your evil competitor does to your website after hijacking the domain, not you yourself, ffs