I know of someone who had a red bump on their arm and asked ChatGPT if that was a problem and if they should see a doctor. It said no so they left it. Turns out it was a tick and they now have lyme disease.
I have a friend who uploaded a photo of a mark on their back to Claude, which stated it was a cut that was healing. Went to the doctor, it was skin cancer.
That said, Claude did say “I’m not a doctor, if in doubt make sure you see one”. So, that’s good… but then, what’s the point in using it?
I feel like I'm reading comments on LLMs writing code where they romanticize that programmers were perfect before LLMs.
Doctors are not infallible. I know plenty of them and they absolutely tell me who and who not to go to. The old joke, what do you call the person who graduated last in their class at med school? Doctor.
Lyme disease in particular has all sorts of issues with diagnosis - from real doctors!
https://pmc.ncbi.nlm.nih.gov/articles/PMC10778834/