But each time I suspected I could have done it better and faster manually
There is a class of tasks that can't be done faster manually, unless you're some sort of colour-smells-like-chicken-and-numbers-have-taste genius. And there is other class (my suspicion now is any non-standard task+framework) that are slower than using agents. So I can imagine you have excellent experience with some tasks like USB hacking and would do it faster than LLM. On the other hand for me, as a Java developer, hacking a USB is finally possible with LLM. Otherwise I'd need to stop-and-learn for some time, which I wouldn't, so either I'd by a more expensive hardware that fulfills my requirements, or put the USB reverse engineering project to my 100 acre todo list
It all depends on the code quality bar. If it's high, a lot of tasks will not be completed much faster. The main speed comes from trusting LLMs output. When you review each change and reprompt LLMs to make the code look like you want. Suddenly, things become much slower and reviews/reprompts are very mentally exhausting.