alt Hacker NewsSeems like a good time to recommend Apache Datafusion. It's designed to be a library, but works pretty well standalone as well. There's a CLI version, but also Python and Java bindings, as well as a Rust library, of course.
From my own experience I can say it integrates far better into your Rust app than DuckDB does.
Well over 100 monthly contributors, too.
Replies
tobilg • today at 2:03 PM
You can even use DuckLake with Datafusion: https://github.com/tobilg/datafusion-ducklake-provider
asa400 • today at 2:17 PM
Is Datafusion related to Polars at all? I ask because you mentioned Rust integration.
➕ show 1 reply
colinmarc • today at 1:37 PM
Came here to post the same thing. DataFusion is fantastic.
Datafusion has all of the individual pieces, especially if you just want to data analysis, but is far from an embedded database.
Indexes, transactions, a first-class storage format are all things that come included with DuckDB, that you won't have with Datafusion.