Databricks started in 2013 when Spark sucked (it still does) and they aimed to make it better / faster (which they do).

The product is still centered Spark, but most companies don't want or need Spark and a combination of Iceberg and DuckDB will work for 95% of companies. It's cheaper, just as fast or faster and way easier to reason about.

We're building a data platform around that premise at Definite[0]. It includes everything you need to get started with data (ETL, BI, datalake).

0 - https://www.definite.app/