> an LLM to sell you an idea that sometimes might be right

An LLM is just an information resource that can be useful when used properly. It can also be useless or net harmful when used improperly. The same is true for web search, libraries and even human experts. A licensed medical doctor is a domain expert and competent domain experts are often correct, but not always.

I don't think it's possible to suggest a universally 'correct' default position on when (and how much) to accept your GP's medical opinion over any alternatives. It depends on too many variables: the context, the person, the alternative info sources, the expected value of correctness and the potential consequences of error. But decision theory suggests this is the kind of combinatorially complex problem space for which any single default position, whether "Always trust your GP" or "Never trust your GP", for every person and situation cannot be optimal.