You don't want other parts of the system thinking that the data is committed to durable storage when it isn't.

"Storage dies" is a broad set of causes. If there is power loss or a server crash due to a multitude of reasons, you want committed transactions to actually be committed.

Having replicas doesn't change the problem. If the orchestrator requires at least N replicas to be done committing the data, you'll have to wait for fsync in N of them.

Without fsync, the data anywhere could be sitting on a write back cache in memory instead of being flushed to the disk.