So have you never had a problem the LLM couldn't solve? What do you do in that scenario?

How do you guide the LLM away from making a horrible choice if you don't understand the internals?