logoalt Hacker News

skeeter2020 • today at 4:25 PM • 0 replies • view on HN

do you have not-big big data* that individuals or sub-teams/groups use? Lots of companies have centralized data <things> (bases, warehouses, lakes, swamps, etc) and runtime security is a challenging issue (i.e row-based security). If your audience measures their data in GB or MB, DuckDB can be an excellent way to punt on the security challenges and just provide them with ALL their data - maybe even locally - while they still get first-class DB tooling and performance (or even better than classic client-server). Example - old way: build a bunch of dimensions and facts for marketing, manage their access and permissions, produce reports, make sure you don't leak data they shouldn't see. new way: write (explore using AI) some extract/transforms from your SoT & create parquet files, use duckdb to serve that up to an individual or group's reporting solution or adhoc usage. IME that one-time investment is not that big and replaces a lifetime of scaling & access issues, or worrying about someone writing a query that doesn't join with the user table correctly!

* IME most companies don't actually have BIG data, but are talking about (at most) a few TB, which DuckDB is fine with.