alt Hacker NewsReally hard disagree. Coming from hadoop, databricks is utopia. It's stable, fast, scales really well if you have massive datasets.
The biggest gripe in have is how crazy expensive it is.
Replies
DebtDeflation • yesterday at 2:13 PM
Hadoop was fundamentally a batch processing system for large data files that was never intended for the sort of online reporting and analytics workloads for which the DW concept addressed. No amount of Pig and Hive and HBase and subsequent tools layered on top of it could ever change that basic fact.
winwang • yesterday at 7:46 PM
If cost (or perf) is the issue, we're building a super-efficient, GPU-accelerated, easy-to-use Spark: https://news.ycombinator.com/item?id=43964505
Spark was a really big step up from hadoop.
But these days just use trino or whatever. There are lots of new ways to work on data that are all bigger steps up - ergonomically, performance and price - over spark as spark was over hadoop.