Well that's part of the problem. AI is not accountable - if you take its advice and hurt yourself, who is responsible?

A real doctor is accountable.

They might both "know" a lot of things but implicitly the party who is accountable is going to be more trustworthy.

And I don't see that going away until AI companies must be licensed for application x and can lose their license / be sued if engaging in malpractice.