> What exactly would the alternative have been in 2006?
Well, WebDAV (Document Authoring and Versioning) had been around for 8 years when AWS decided they needed a custom API. And what service provider wasn't trying to lock you into a service by providing a custom API (especially pre-GPT) when one existed already? Assuming they made the choice for a business benefit doesn't require anything close to a conspiracy theory.
And it worked as a moat until other companies and open source projects started cloning the API. See also: Microsoft.
WebDAV is ass tho. I don't remember a single positive experience with anything using it.
And still need redundant backend giving it as API
WebDAV is kinda bad, and back then it was a big deal that corporate proxies wouldn't forward custom HTTP methods. You could barely trust PUT to work, let alone PROPFIND.