I'm in the process of migrating away from my custom domain on Google (I'm grandfathered in from back when they offered it for free). So far getting all my old email, contacts, and calendar info FastMail has been seamless. All that's left is to repoint the DNS.

Like another comment, I too have a couple of services connected to "Sign in w/ Google". Until I actually delete my Google org, those will still remain active. Since I'm not paying, I feel OK about this.