Going to a probabilistic system for something that can/should be deterministic sets off a lot of red flags.
I’ve worked medical software packages, specifically a drug interaction checker for hospitals. The system cannot be written like a social media website… it has to fail by default, and only succeed when an exact correct solution was determined. This result must be repeatable given the same inputs. The consequence is people die.
a drug interaction checker can be deterministic, based on a static corpus of drug interaction data
a diagnostic system should not necessarily be deterministic, because it always operates on incomplete data and it necessarily produces estimates of probability as an output
Despite this, given that health care prices in the USA continue to accelerate from relentless government interference and regulation, perhaps if the deterministic side of things could be figured out, we could monumentally decrease cost and increase access.
Health and medicine is very far from deterministic. Your drug interaction checker is deterministic because the non-determinism is handled at a higher level (the doctor / patient interaction) in the health care system. Individual patients often respond wildly differently to the same medicine even in the absence of drug interactions.
Similarly the non-determinism in ChatGPT should be handled at a higher level. It can suggest possibilities for diagnosis and treatment, but you should still evaluate those possibilities with the help of a trained physician.