I agree about data backups but replicating your setup in another cloud provider is:
1) Expensive
2) Not straightforward, e.g. is there a 1:1 setup in another cloud for your system?
3) Likely to go untested and be useless when you need it most
The best part of cloud providers is that short-term VMs are relatively cheap to deploy. You don't need a full active-active failover setup, you just need to design your infrastructure in a cloud-agnostic way and test the deployment scripts a few times a year.
The most expensive part is going to be maintaining an up-to-date offsite data backup. Running a few VMs for a handful of hours is basically free.
I do think (1.) depends on your company size, and business model. For most, it's cheap, e.g.
https://rsync.net/pricing.html
That said I was once a CTO for a company with 10 photo studios and we had a large amount of new (raw, DSLR) photos per minute, so cost was an issue and also upload speed for offsite backups.
Fully agree, that's why you need to think well first and come up with a compromise that you are willing to accept. Periodic testing of your DR procedures is non-negotiable but fortunately it's usually much simpler for smaller startups than for larger orgs.