That is indeed the problem. And when you have to meticulously check everything that the LLM does (because you can't know where the hallucinations will be), it completely destroys any productivity benefits you would gain from having it write the code in the first place. Thus you wind up going no faster (if not slower) than you did in the first place. The only way to go super fast (the way some people claim they are) is to discard quality.

As has been pointed out over and over: the time consuming part of programming was never typing code into the computer, it was understanding the problem and the logic behind the code. Using an LLM only addresses the fast and easy part of programming, not the hard parts.