As someone who's been doing AI call centres for a while, I'm not surprised. It's really, really hard to build voice AI that works. There are no open source solutions that work out of the box.
But the biggest problem is ASR. WER is still atrocious even with SOTA models. When you add drug names and regional accents, it's a recipe for disaster.
Just curious, why are drug names such a disaster? Do LLMs not understand pronunciation keys?
ASR = Automatic Speech Recognition
WER = Word Error Rate
Too many acronyms; not everybody is in your field.