logoalt Hacker News

potamic11/08/20241 replyview on HN

Would love the authors to pitch in with their use cases, but I think most people simply do not need sub millisecond analytics. This is mostly replacing typical spark pipelines where you're okay with sub second latencies.

S3 is the cheapest, fully managed storage you can get that can scale infinitely. When you're already archiving to S3, doubling it for analytics saves cost and simplifies data management.


Replies

hipadev2311/08/2024

Sub-millisecond? Literally nobody in large data analysis is doing that. Can you cite some sources or show me what sort of setups (with any DB tech) that you're able to run meaningful queries on S3 sources with even sub-second latency?

show 1 reply