S3 Is the Future, S3 Is the Past

One thing I find notable about S3 today is that, while it used to drop in price reasonably often, there hasn't been a price drop in a full decade:

  2006-03-14  $0.150/GB-month
  2010-11-01  $0.140/GB-month
  2012-02-01  $0.125/GB-month
  2012-12-01  $0.095/GB-month
  2014-02-01  $0.085/GB-month
  2014-04-01  $0.030/GB-month
  2016-12-01  $0.023/GB-month

Right now it is not even clear how to interface with SSDs even on a single host, there has been all sorts of attempts to move away from the traditional plain block device model. NVMe has extensions for KV, ZNS, and FDP, all which offer different characteristics. And then there are of course open channel SSDs and some others too. I kinda expect the future foundational IO interface to be more S3-like than block-device or unixy filesystem-like.

Nice article. I agree that it is a bit of a shame that everything is forced to be so S3-centric (and I say that as someone whose work helped motivate a lot of people to do that), but right now its an unfortunate reality of running software in the cloud because cloud networking and SSDs are so expensive that you really are required to use S3 if you want a system that can handle "big data" scale workloads cost effectively

The unfortunate part of that graph is that I think it's not updated for today's prices given the memory shortage/crunch we're experiencing that's driven up the prices for all kinds of memory.

But also while it should be faster than it is, how would an S3 designed around SSDs look differently to an API user? I would think the API is basically the same.

We ran EBS for years. Finally the costs became ridiculous and we moved our entire dataset to S3. We’re saving 20k/month. There’s just no beating the price.

I find Cloudflare R2 to be much cheaper as there are zero egress fees irrespective of data size. One is only billed on the count of requests with 10 million free download requests per month (check pricing for precise info).

If you do a clever bit of caching work in your app like we have with our apps Slyp and SlypBusiness, you could even make it insanely cheap.

We have a file manager that is ultimately responsible for images, files, videos and whatnot. It loads the images/files from the downloaded local cache when requested by a feature in the app.

A standard feature uploads and downloads using signed urls obtained from the backend service. The manager increments the file's version in the backend on each successful upload. For download, the manager compares the version against the locally cached version. If there is an update, the new file is downloaded in the background and overwrites the local cache and notifies all features using the file inside the app.

> S3’s dominance is due to its many advantages: effectively infinite capacity, high durability, and low per-gigabyte capacity cost.

It's not "cheap". 6 months of S3 is at around price of outright buying 4TB SSD at retail price. That before you do any IOPS to it

S3 is terrible deal on any front. it's just easy