With pharmacies, people are emotional because their perscriptions getting filled can literally mean their livelihood, their health or their life. I hoard medications because my insurance will only cover 30 day fills, and often it takes 2-4 days to call enough people for insurance to cover a refill. Its a common problem for older folks to take half of their perscribed medication from either cost, or inability to get through to the pharmacy.
You effectively have AI gatekeeping healthcare, as companies roll these out and also take away any escalation path that a human call center, or local pharmacy staff might've been able to handle.
I've been also working with AI for years now, and it is absolutely horrifying to me that any company would feel comfortable rolling it out - you're basically saying a 5% error rate is acceptable to you when you're providing a gatekeeping service that is life or death.