This is not client side. Images just sitting on your Android phone are probably safe (although Google could push an update at any time). Your images get scanned when you back them up, send them over RCS, etc. I have even seen criminal cases originating from reverse image search - anything that touche the servers of the big tech companies, except apparently Apple, will be scanned using questionable AI and against a secret list to Protect the Children.
This is an image that he did not send off of his device to anyone except his doctor's office, yet Google reached into his private data and scanned it anyway.
Google reported him to the police based on a single false positive.
To add insult to injury, even after the police contacted Google to tell them that they had cleared him of wrongdoing, Google refused to restore access to his account.