That's exactly the point of smart contracts.

The contract can hold the money in escrow such that it can only be sent either to the seller or returned to buyer.

The seller and buyer can then both walk the contract through a state machine on agreement (i.e. confirm shipping, confirm delivery, potentially also confirmation for a return process) and when the buyer and seller come to a disagreement (ex: seller attests they've shipped the product and it should be delivered but the buyer asserts they havent/the tracking on shipping is invalid) or one of the participants is non-responsive for a certain amount of time then the contract moves into arbitration.

In arbitration one or more third parties then step in to serve as arbiters/oracles that decide in the favor of one party or the other and commit those decisions to the contract and the contract then derives consensus from those decisions and proceeds to the corresponding state/action of the contract (i.e. refund vs close).

Now your arbiters/oracles/third parties have reputations and you can reason about how trustworthy they are before you enter into the contract.

This means all parties can evaluate their risk tolerance and trust levels before entering the contract/on agreement.

-------

TLDR: Trust is inherent to any system reliant on the physical world. The point of smart contracts, etc is to formally encode those trust assumptions and the procedures of the contract in as trustless of a way as possible and to allow distribution of that trust across parties with most of the coordination overhead being automated/abstracted away.

And importantly smart contracts provide an extremely low friction happy path. In the happy path where all parties are satisfied, it's extremely efficient and responsive. But in every other path, the conflicts, incentives, and resolution procedures are clearly defined for all parties involved.