logoalt Hacker News

jtbakertoday at 4:12 PM0 repliesview on HN

I've got a couple of different use cases:

- ETL pipelines running on K8s nodes. Using their streaming processing engine means I can run smaller pods/nodes if needed, for datasets that may have required large dataframe-like transformations that may have buffered a big dataset into memory previously.

- A CLI distributed to an internal team to do a postprocessing step on a large modeling dataset - to get it into a consumable format and upload it to a bucket as a .db file.

- A SvelteKit app that used the node duckdb bindings to attach to the .db on the bucket and explore the results through a suite of BI tools. These tables have millions of rows, and would be pretty heavy to store in PG. The DuckDB version works really, really well.