What I've found is that most people who dislike the chat interface aren't using it in a way that leverages its strengths.
Up until recently, LLMs just plain sucked. You'd set them on a task and then spend hours hand-holding them to output something almost correct.
Nowadays you can have a conversation with the chatbot, hash out a design, rubber duck and discuss what-ifs until you have a solid idea of the thing you're building, codified in a way an agent could understand, and now you have a PLAN.
From there, it's a matter of setting the agent in motion and checking from time to time to make sure it's not getting stuck on something under-specified.
That said, I've found that this kind of workflow works a lot better with claude than with gemini.