I mean most taxes like this have an 'above X amount' clause. Such as the gains you get taxed on when selling your home. California it's $500K in gains if you are married so extrapolating that your scenario would be covered.