It is worth the exercise to honestly assess the maintenance, initial costs, ongoing costs, etc etc for owning your own. I did this at a company for a single server to let the ml group test fine tuning frameworks/throw crazy ideas at before spending real money on a training run and it paid for itself in like 3 months. Sometimes the math is overwhelmingly positive, sometimes it isn't. In general, the smaller/earlier you are the less it makes sense. Not because of the per hour costs, but because of the distraction away from implementing the core idea of the company. It is worth some, maybe a lot of, inefficiency to stay focused. When you get bigger and have strong fit and a clear direction it is a lot easier to focus on optimizing hosting costs and spending leadership time away from the core problem of the company.

Oh man I cannot tell you how many times I've gotten into arguments on here and other technical forums years ago about self-hosting and how 'stupid' I was to not want to rent cloud servers

You’re expressing a very narrow engineer’s perspective that doesn’t consider the realities of managing bare metal hosting at any kind of scale.

These are business decisions, made in terms of core competencies, capex vs. opex, and the difficulties and cost of building out a reliable, sustainable hosting operation that handles all the compliance and security requirements, and the full range of “ilities” that real businesses have to deal with.

The fact that Bob in IT might be capable of doing some of this on his own doesn’t really enter into the picture. It’s not relevant.

There’s a reason that most companies don’t operate their own electricity generation systems. Much the same is true for computing systems.