It's correct, by some definition of "correct", essentially all the time. It always produces text that matches the probable token distributions that it was trained on. You seem to want it to be an oracle. Your job as an engineer that works with them is to figure out how to make what it _actually_ does useful.
That’s just a lot of words to say “if you don’t like the results that’s your problem” which is, quite frankly, ridiculous.
AI gets all these passes while also apparently being this amazing perfect 10x productivity multiplier. If you squint yeah you can see it maybe. But you can’t squint all the time and I won’t pretend I’m not squinting.