> Those consequences are the reason that most database systems are built for either transactions or analytics, and not both. [24]
> [...]
> [24] People have tried to build one system that does both in the past (HTAP, or hybrid transactional/analytical processing), but those attempts generally didn't work out. Some newer systems seem to be bridging the gap, however. CedarDB, the commercial version of Umbra (a research system from TU Munich), runs transactions and analytics on just one engine and one copy of the data. Databricks' Lakebase keeps Postgres (a row store) as the engine for transactional queries, but its storage layer rewrites the same data into columnar formats in the background, allowing an analytical engine to query it.
The footnote says "those attempts generally didn't work out" but then mentions Lakebase and CedarDB as newer efforts. Can anyone speak to whether or not CedarDB, Lakebase (and any others?) are achieving the fabled transactions + analytics in one database? Should we expect, at some point in the near future, for transactions + analytics to be standard in all databases, or is combining them a niche / esoteric thing with many tradeoffs?
I think there's never going to be a point where you can have the kind of performance you can get by optimising a datastructure for a 'fixed' dataset. Nor can you get the maximal performance updating data if you need to maintain a useful datastructure.
A simple case is a sorted list, you can sort it after every update, you can ignore the ordering and sort it as needed, or you can use a datastructure that is typically O(log(N)) at both updates and retrieval. Better datastructurs help, but they cannot remove the tradeoff entirely (in fact O(log(N)) inserts are kind of theoretically minimal, though a trie can do it in O(1) for fixed length datatypes).
There's also TiDB which has decent adoption. It allows for columnar replicas of transactional data and exposes both over the same MySQL-compatible interface.
[flagged]
There are always tradeoffs. Databases are fundamentally about managing optimizations, and the question is whether those tradeoffs make sense for your workload.
HTAP has historically been niche because the cost of analytics on fresh data wasn't worth the squeeze (sooner or later the MBAs step in). Separating the OLTP and OLAP layers gives you better workload isolation and the ability to optimize each independently.
Databricks Lakebase/LTAP uses Postgres for transactions and lakehouse engines for analytics, with a shared storage layer. They also use Spark to build Postgres pages in parallel and write them directly to storage, avoiding the single-writer bottleneck when loading large volumes of data.
SingleStore takes a different approach: a distributed database based on MySQL with an in memory rowstore for transactions tightly coupled with a merger to columnstore for analytics.
I don't think HTAP DBs will become the default for every use case. But I do think the space will grow as we move towards agents and automated decision making and optimizations.