alt Hacker NewsThere are self hosted object stores which use the same protocol as S3. One example: https://github.com/minio/minio
Parquet files are smaller than row based storage in a database (but not those databases with focus on strong compression).
And for backup - the files are probably easier to just copy to multiple disks for redundancy, as opposed to database dumps and incremental backups which at the Petabyte scale will be a pain.
Replies
your reply makes zero sense.
there is a reason why people develop for S3: a lot of enterprise data is there. people ingest there from various sources. and it's not just parquet usually, it's multivendor sources writing to an iceberg catalog.
nobody will run minio on AWS other than hobby projects and small demos.
I regularly work with iceberg datasets in the double digit TB range per dataset. keep that in mind when you think about sizes. databricks, snowflake, large enterprise vendors: they are targeting these sizes.
Minio is no longer maintained.