Just use the damn file system. Why does everyone have to put HTTP between everything?
I can pay Amazon pennies on the dollar to saturate 100Gbps with arbitrarily high concurrency
Sometimes I also want a smaller version for when I don't need that performance but don't want multiple implementations of storage backends
Edit: before someone says NVMe, highly available across 3 fault domains
I like how you got 4 different comments all starting with "Because..." hah!
Because AFAIK the filesystem is at the same time a huge API and designed for a different use case. I can think of the following examples:
- very flat structures: storing hundred of thousands of files in a single directory will fail
- designed for local disks and NFS is a leaky abstraction:
- running a system using on locks will fail
- cache behaviors work fine for the "humans browsing files" usecase but not so much for other cases
probably many more reasonsBecause if you ship a tool with s3 support, the customer can decide if he wants to store in the cloud, or spin up a docker container locally.
Because you want to mock the tools you are going to use in production locally or in CI. Really good reasons to not totally change what you are doing in your app between environments.
Because you might be running something that only talks S3 and want to point it at something local.
Because this is about having a local standin for S3, which (in a production deploy) often solves a different problem than a (local) filesystem.
This post does not even mention the S3 API as a replacement for a filesystem.