What I would really love is a dead simple way to: 1) connect to my transactional Postgres db 2) define my materialized views 3) have these views update in realtime 4) query these views with a fast engine
And ideally have the whole thing open source and be able to run it in CI
We tried peerdb + clickhouse but Clickhouse materialized views are not refreshed when joining tables.
Right now we’re back to standard materialized views inside Postgres refreshed once a day but the full refreshes are pretty slow… the operational side is great though, a single db to manage.
that's been supported in striim since 2016
we have the exact same problem. We tried Clickhouse but their materialized view limitations stopped us.
Yeah you definitely need something like Feldera: https://github.com/feldera/feldera
You can do this with Materialize or Feldera. The keyword to look for is “incremental view maintenance”