My few cents:
- Compute and storage separation simplifies managing a system making compute "ephemeral"
- Compute resources can be scaled separately without worrying about scaling storage
- Object storage provides much higher durability (99.999999999% on S3) compared to disks
- Open table formats on S3 become a universal interface in the data space allowing to bring many other data tools if necessary
- Costs at scale can actually be lower since there is no data transfer cost within the same region. For example, you can check out WarpStream (Kafka on object storage) case studies that claim saving 5-10x