alt Hacker NewsFor now, BemiDB supports only full Postgres table data re-sync. We plan to enable real-time data syncing from Postgres into S3 by using logical replication (CDC), which is much more reliable than PG NOTIFY.
We use logical replication and this exact approach with our other project related to auditing and storing Postgres data changes https://github.com/BemiHQ/bemi. We're thinking about combining these approaches to leverage scalable and affordable separated storage layer on S3.
Lmk if that makes sense or if you had any more questions!
Really interesting thanks! I guess my use-case would rather require incremental updates
Ideally it would just sync in real-time and buffer new data in the Bemi binary (with some WAL-like storage to make sure data is preserved on binary crash/reload), and when it has enough, push them on S3, etc
Is this the kind of approach you're going to take?