alt Hacker NewsIt is interesting to me! Do you know if the above applies to Datastream for CloudSQL as well, is it the same thing, or are you talking about a more DIY approach to CDC here?
I assume not everyone using MySQL and BigQuery are using CloudSQL and so don’t have Datastream available to them.
FYI, Datastream works with pretty much any MySQL instance, including self-hosted and even AWS RDS and MariaDB. See: https://docs.cloud.google.com/datastream/docs/sources-mysql
Datastream solves the issues mentioned in the article, because it relies on CDC, which is the same solution the article is proposing.
Note that the article seems to have a major error in its first sentence: “MySQL CDC syncs miss deletes and intermediate updates.” CDC is a solution to that issue, it doesn’t suffer from it. The second paragraph correctly describes the approach that has limitations: “a scheduled job selects rows…”