3PLs and warehouse distribution channels have gotten better and are cheaper than using FBA shipping outside Amazon.
Did you know the biggest sellers on Amazon normally have their own website and in fact, orders from Amazon flow directly into their OMS (order management system). Amazon is simply a sales channel. Also Amazon has a stronghold in western countries but much less so in gulf states, South Asia, and Africa.
Can you give some more information for buyers who want to route orders directly to sellers instead of via Amazon? Right now the only benefit for using Amazon is a unified shopping interface and free shipping using Prime.
Amazon's biggest feature is free returns. I would not buy even a quarter of the stuff on Amazon if I was not confident that I could easily return it.
As someone who ran part of their fulfillment network, I say you are completely off the mark here. Their network is the best in the world bar none. In addition to marketplace, 1P, and FBA, there are many other cooperation models where massive companies that appear to have their own distribution centers actually use Amazon's network (even co-located for some warehouses/sort centers).
There is no 3PL or other warehouse companies (many of which use Amazon's anyway) there are better than Amazon. Maybe Walmart gets close. For the case you mentioned (OMS integrations), this absolutely does NOT mean that they handle shipping. The aggressive majority of those vendors just uses FBA anyway.
If you're building your startup on the principle that Amazon is "a sales channel only", then I wish you well but you will fail.