logoalt Hacker News

huntaubyesterday at 10:43 PM0 repliesview on HN

Our product is Archil [1], and we are building our service on top of a durable, distributed SSD storage layer. As a result, we have the ability to: (a) store and use data in S3 in its native format [not a block based format like the other solutions in this thread], (b) durably commit writes to our storage layer with lower latency than products which operate as installable OSS libraries and communicate with S3 directly, and (c) handle multiple writers from different instances like NFS.

Our team spent years working on NFS+Lustre products at Amazon (EFS and FSx for Lustre), so we understand the performance problems that these storage products have traditionally had.

We've built a custom protocol that allows our users to achieve high-performance for small file operations (git -- perfect for coding agents) and highly-parallel HPC workloads (model training, inference).

Obviously, there are tons of storage products because everyone makes different tradeoffs around durability, file size optimizations, etc. We're excited to have an approach that we think can flex around these properties dynamically, while providing best-in-class performance when compared to "true" storage systems like VAST, Weka, and Pure.

[1] https://archil.com