alt Hacker NewsKaiser nurses say AI, workplace surveillance are making their jobs, care worse
Comments
If you think using a machine to evaluate how well a human is showing empathy is a good idea, you probably shouldn't have any position of power.
That is surprising. My primary care provider had a different response. Basically he said something in line of
"You wouldn't believe how much of a relief it has been. In your last visit, you saw me typing everything you were saying, right? I don't have to. I can listen to you and take very specific notes as necessary as opposed to focusing on both typing and listening to you at the same time. It has bought my stress levels down to here." (Indicated by his hand lowering)
Workplace metrics and surveillance... What could possibly go wrong?
That being said, more than one (female at that) doctor has told me in confidence and based upon their observations during residencies, etc. that if I'm ever admitted, be very careful how I modulate my interactions with nurses. They're not all Florence Nightingale and Mother Theresa and there exist those who will @#$% you up on various pretexts or are just plain sloppy and negligent.
Despite all the moralising fluff, it's just a job, not some saintly vocation. Some safety oversight is needed, just as it is for any other work function. Still, can bet that anything 'Corporate' has mandated will be Goodharted up the wazoo.
This problem with workplace AI interfering with nurses ability to manage healthcare obligations for their clients is not confined to Kaiser. UHC has also introduced the AI surveillance tools and are using it to do similar things.
The article mentions uses of AI but doesn't really give any examples of harm from AI. It does give specific examples where it sounds like Kaiser is optimizing calls to minimize cost rather than improve quality of care. It's also pretty easy to expect that the examples given (treating longer calls as a problem and penalizing nurses for giving more than 3 pieces of advice) reduce the quality of care.
Kaiser has priorities, healthcare being #4 or 5
The nurses at our local rural hospital are tagged and tracked wherever they go on the hospital campus. Time spent in one spot is part of their review.
I wondered why they zip in and out of the rooms, when just a few years ago they would spend fifteen to twenty minutes in each room. The patient load hadn't grown. The number of nurses has gone up, not down.
So I'm blaming the stupid metric on their evaluations for a worse standard of care.
Kaiser was the first insurance company in america. If they cant make it work then we are lost.
There is an unfortunate cycle that has been playing out this year in USA. Insurance companies are arming up with AI to deny billing codes charged by providers. Providers are arming up with AI to listen in on provider-patient sessions to prove the billing codes are legit (And teach providers to use keywords in session).
The loser, as always, is the patient's quality of care.
Thinks that are not allowed in the EU thanks to the AI Act.
It sucks because AI reduces call center costs in many toil tasks very effectively, regardless of whether you want to help or dissuade customers - it's just well-suited to the task and it's going to accelerate. It's a win for capital and the technology implementers, even if it's a loser for the call center employees and callers.
Some of the tech is pretty scary. One big vendor's solution [0] can provide not just AI agents but also use AI to snoop on calls in progress, evaluating sentiment from both sides [1], verifying phrases are said - pretty dystopian in theory. From experience, these things tend to go downhill based on the attitude at the top - is the mission to slash costs or take care of customers? A 1000 decisions follow from this one, and like Jira, it can be a useful tool or a prison-like hell.
[0] https://www.cisco.com/c/dam/en/us/products/collateral/contac...
[1] https://www.cisco.com/c/en/us/products/contact-center/webex-...
They want you to blame "AI" like Claude jumped up and decided to start hassling nurses, not individuals in management
> "Nurses fear that having long calls can lead to bad performance reviews"
> A company spokesperson said, "Kaiser Permanente does not use Average Handle Time to assess agent performance"
So uh, average time wasn't raised as a concern, calls beyond a certain threshold was. I wish this semantic discrepancy was better highlighted in the article.
I'd like to think it's not meant to make life better for a regular nurse, but rather to weed out the abusive ones? At least that's how I see it from my flawed, post-communist country perspective.
AI is making lots of jobs and workplaces worse.
Every time “relentless push for efficiency destroying X” comes up I should repost this:
https://sohl-dickstein.github.io/2022/11/06/strong-Goodhart....
This is a well understood phenomenon.
the more I read about how AI is being used, the more I'm coming to the conclusion that despite me personally find it quite useful in my work, and even my personal life to some extent, it is a net negative to society as a whole, and if the choice is between zero AI or the AI that is being deployed, we'd be better out without it
what significant improvements to society or humanity have come about as direct result of AI, that wouldn't have been achieved without it? faster protein folding is the only one I can think of and that more a matter of "faster" than "impossible without AI"
I think at some point there's going to be some version of the Butlerian Jihad
Obligatory dystopia reminder. It doesn't have to be this way https://marshallbrain.com/manna1
Kaiser has some of the best healthcare outcomes in the country/world due to their protocols and how good they are in ensuring adherence to them.
It’s going to be very improbable that these statements are true.
https://www.youtube.com/watch?v=DpQamTrynBY
[edit] preemptively, if you're going to claim "but this isn't art so irrelevant" then I claim bull fucking shit. It's the same problem no matter how you slice it whether it be engineering, support, art, or medicine. Get real. Look inward. Touch grass. If you think AI is a good thing for your profession--whatever that profession might be--you're probably a delusional psychotic. It's OK, you'll thank me later.
[flagged]
There's a reason that among Americans, telekinesis and creationism are more mainstream positions than sloptimism.
About 14% of Americans think AI is moving us towards a better world. About 17% are creationists. About 26% believe in Telekinesis.
I would love to learn the cost of the AI versus how many new workers they could afford to hire and get more calls done. But I assume the end goal will be full replacement of human workers once the AI has had enough time to learn the job.
Unpopular opinion: People want to be lazy and hate things that force them to work harder.
They will vocally rationalize it.
I did it. "I'm more productive work from home." But then I do dishes, take an hour break, paid.
Foucault says that when people are observing them, power is placed over them.
If you are a worker you should hate this.
If you are a customer or owner, you should like this.
But I certainly won't be automatically believing people under surveillance who make claims it makes their quality worse.
> Nurses are instructed to stick to a script on phone calls and give no more than two to three pieces of advice, Capulong and other nurses said, which means they may sometimes need to decide whether to withhold advice or face a performance evaluation hearing.
> Another nurse speaking on condition of anonymity said “AI did not understand our job and would grade us wrong all the time.”
It's always worth remembering Goodhart's law https://en.wikipedia.org/wiki/Goodhart%27s_law - "When a measure becomes a target, it ceases to be a good measure."
In theory AI could usher in the first time in history where one can escape from this trap - because qualitative judgments can be made at scale, from an unbiased and universal baseline. In this situation, for instance, rather than collapsing call transcripts and reports into metrics, it could evaluate whether red flags are encountered in the context of a call, and allow for qualitative guidance on improvement, across a comparative corpus of situations that are themselves chosen qualitatively.
But very few managers are empowered to take this kind of approach; they're evaluated by their ability to report quantitative metrics, and thus they must implement regimes of quantitative metrics. And leadership instructs them to use AI to build that regime more quickly.
If you want to see an "AI native" organization, it's one where leadership actively fights this tendency, and sees managers as product designers who make the end-user experience a beloved and empathy-driven one, as opposed to a gear that turns accountability into a single number on a screen.
Just think -- with the increased efficiency, they will have more time for Tik Tok dances.
What these complaints always boil down to is autonomy and control. The more centralized an organization, the more it relies on metrics to understand and exert control over its employees and customers.
People started hating tech right around the time metrics became popular. I don't think it's a coincidence. AI just accelerates the trend.
The problem is the misidentification of AI as the issue. As long as we don't understand the real issue, we won't solve it. AI is just a tool. It's being used in a way that denies human agency.
Our cultural values need to shift away from safetism that demands centralization. And shift toward valuing human agency. That starts with talking about the core issue.
While I'm sympathetic to the frictions of newly introduced AI and the fact that AI in healthcare, especially calls, can seem very uncaring, between the lines the article reads a bit like the typical union complaining about modern tech that reshapes their work, given the multiple mentions of protests, nurses union, etc.
Given how healthcare is one of these sectors that seems to relentlessly resist efficiency increases and is the prime example of Baumol's cost disease, I think any developed country with a costly healthcare system needs to do these AI experiments. The current versions will be shit, but the only way out is through if you still want to provide affordable care.
I honestly have no doubt that AI going forward will be able to do a good job at triaging via calls and also being empathetic about it. But of course it needs careful experimentation.
I RFTA and the majority of the complaints are about call center metrics and the pressure to ration care. These are real concerns about misuse of metrics, but not AI. The AI empathy thing was a 2024 pilot that was discontinued.
FWIW my wife works for Kaiser and finds a lot of value in the the medical LLM tools available to her. She tells me being able to do live translation, summarize notes, and quickly get comprehensive answers save her time and help her give better care. Her older patients also frequently come in bringing AI-powered alerts from their apple watches that detected cardiac events.
It's annoying that we use broad terms to describe a set of technologies that in some ways can be problematic and in another ways are very beneficial. We gotta evaluate each of these as they come rather than talk about blanket bans.