logoalt Hacker News

WalShadow: Sub-second Postgres replication to ClickHouse from physical WAL

47 pointsby spathaklast Friday at 4:09 PM6 commentsview on HN

Comments

vmspyesterday at 9:14 PM

They're not using `wal_level = logical`, which has been the "friendly" way of doing CDC on Postgres since ever, but are going straight to `wal_level = replica` which, afaik, has never really been used to build something atop of except Postgres' own replication.

This is very interesting. I'd never have guessed that it'd make such a difference. I also bet this is the sort of thing that would have never end up being implemented without access to coding agents. Having to figure out these protocol-level details is no longer the huge time sink it was

show 2 replies
rgbrgbyesterday at 9:48 PM

would this work with a postgres hosted on supabase? for our company brain mcp thing we mirror postgres data (along with server logs, banking data, slack, etc) in clickhouse but took a very inefficient polling approach since it works with any read-only db connection string. always looking to improve that and get rid of latency though.

show 2 replies