It’s a pretty fine line, and if they want to do anything beyond just telling you things about your data, actually taking any sort of actions or communications with health care providers, they should qualify as a business associate under HIPAA. And I certainly am never going to share my healthcare data with an AI company without any sort of regulations or oversight, beyond a TOS.
A Business Associate (BA) under HIPAA is a entity that contracts with a Covered Entity (CE) using a Business Associate Agreement (BAA). OpenAI wouldn't be considered a BA unless they had an executed BAA with a CE.
There are a wide variety of things, including communication with providers, that OpenAI can do without being regulated by HIPAA. The provider, however would most likely be covered, so they would be under an obligation to not disclose any PHI.
As an example, consider a hypothetical "email my doc" app. This app could take health information from the patient and email it to all of their doctors without falling under HIPAA at all.
What you're probably thinking of is the HIPAA Privacy Rule (there's more to HIPAA than just that, the Security Rule is pretty involved). Under the Privacy Rule, CEs and BAs can not disclose PHI without a patient's consent, with a few exceptions.
If the patient is disclosing that information though, all bets are off (aside from state level regulations).