This is it right here. I've long thought about this one and whether I should bother with an AI agent that can do all of this stuff for me, but the reality is both what you said and I'm not rich enough.

Do I want the AI Agent to take my bank account and automatically pay some bill every month in full? What if you go a little over that month due to an emergency expense you weren't prepared for? And it's not a matter of "I don't have enough in my bank account for this one time charge", but it's "I don't have enough in my bank account for this charge and 3 others coming at the end of the month." type deal.

Agents aren't going to be very good at that. "Hey I paid $3,000 on your credit card in order to prevent you from incurring interest. Interest is really bad to carry on a credit card and you should minimize that as much as possible." Me: "Yeah but I needed that money for rent this month." Agent: "Oh, yeah! I should have taken that into account! It looks like we can't reverse the charge for the payment."

Yeah, no fucking thank you LOL.