I mean, the LLM was right in both cases. And that's what tests are for: testing whether a hypothesis was correct. It's only a waste of time and money if the LLM is only marginally better than flipping a coin. It's my experience that the LLMs are far, far better than that threshold.

The difference between an LLM and WebMD is that WebMD is a page of information. Any reasoning is yours, based on a small fraction of the information available to you. An LLM is based on all the medical literature of all time. It's not even close to being the same thing.

Even if you argue that an LLM is just a prediction engine, that's kind of exactly the right tool for the job. "If you have these symptoms and not these other ones, these are likely causes" plays against the lone strength that LLMs' detractors argue they do well.