alt Hacker NewsFWIW I know of a pharmacy network that's implementing AI. Not only does the AI make egregious errors like speaking in Spanish when the conversation is in English. It also has a limited context window for ground rules of 5000 tokens or less. So correcting all the various errors goes over the instructions limit. As a result if the implementation goes through it will be of poor quality. All I can think is that people with no technical expertise are making these decisions, it is easy for any techie to predict that AI of this level won't be capable.
Replies
This is an underappreciated point when people talk about AI/chat integrations. If they put a claude-level model behind it, I'm sure it could resolve 99% of the situations faster and better than a human could, or at least realize its limitations when it can't. But of course that is too expensive so they often use extremely weak models that are literally worse than a good incremental full text search box.
This is probably (if the business here is large enough to support it) an excellent area for a tailored model instead of a general purpose LLM. A lot of the context around medication comprehension is going to be rather universal to pharmacies and other HCPs so ideally that information would be baked into the model itself with the actual user supplied context mostly being focused on business specific logic. The commodity LLMs are just an amalgam of a hammer welded to a screwdriver, a car, a compass and a toothbrush - more specialized models, with more specialized testing and offered by a company specialized in the field would possibly offer a decent customer experience here?
With something like healthcare though I'll remain eternally skeptical that anything other than a licensed expert will be palatable to the public.