alt Hacker NewsThe future of everything is lies, I guess – Part 5: Annoyances
Comments
I don't need to conduct 1000 transactions per day. I don't forsee a world in which it will be some sort of fatal inconvenience to need to approve all purchases. I certainly don't plan on ever just handing over my credit card to an LLM, due to its fundamental architectural issues with injection, and I still don't anticipate handing it over to any future AI architecture anytime soon because I struggle to imagine what benefits could possibly be worth the risk of taking down such a basic, cheap barrier.
All that stuff about support, though, inevitable.
My father just changed his car key battery with the help of ai and he likes that. He also consulted it about about car insurance regulations and he got more out of it than searching the web himself.
For most simple mainstream questions I just ask ai instead of googling shitty results.
Most of the time ai is good enough and often better than the status ante.
People do not care if it is a stupid token prediction machine as long as the job gets done.
> Perhaps we’ll see distributed boycotts where many people deploy personal models to force Burger King’s models to burn through tokens at a fantastic rate.
Given how many people hate AI in general, I'm surprised there hasn't been anything like this happening. They could even get around the irony of using "AI" themselves, I bet low-tech language models like Markov chains could provide sufficient time wasting potential (I'd love to see it done with an old fashioned AIML chatbot). Asymmetric chatbot warfare.
The worst thing is that non-technical people, and actually a lot of technical people without experience in ML, will tend to overstimate the capabilities of those systems, neither the nuances of probabilitic thinking to properly integrate their outputs in a decision.
Remember that the polygraph still exists, now we will be dealing with a massive portion of the decision makers will treat as artificial inteligence not in the technical sense we use, but as real inteligence, maybe even super-inteligence.
The erosion and further diffusion of responsibility is the trend that worries me the most, since it’s already how all mid-size organisations, businesses and institutions alike, operate by design, and LLMs are likely to make that much worse.
have worked closely with customer support teams, can confirm that the goal of any technical improvements that go in front of CS agents is to reduce ticket volume, and thus costs. of course they measure retention and satisfaction but ticket volume is always the big one. chatbots were big for this long before LLMs existed.
a fun side effect is that CS is also an early warning system for companies, so when you make it harder to get through to a human, you start throwing out info on your users' pain points. of course this only matters if people have a choice about whether to use your product, so that's gotta be an upside for insurance companies, etc.
A COMPUTER CAN NEVER BE HELD ACCOUNTABLE
THEREFORE A COMPUTER MUST NEVER MAKE A MANAGEMENT DECISION
—IBM internal training, 1979
It took me a while to realise that the premise is saying the same thing as the reason why we have so many "Computer says no" experiences today.
The conclusion only follows if you want someone to be accountable.
If you want to avoid being accountable, computers should make all management decisions. This has nothing to do with AI other than it provides another mechanism to do that.
People saying "I'd love to help you but the computer won't let me do that" has been happening for years now.
Websites develop abusive patterns because A/B testing lets a process decide based on the goal you want, It doesn't measure the repercussions so you have made no decision to allow them.
Management read it as
A COMPUTER CAN NEVER BE HELD ACCOUNTABLE
THEREFORE THERE CAN BE NO LIABILITY IF COMPUTERS MAKE ALL MANAGEMENT DECISIONS
I've been enjoying these articles by 'aphyr and I think they raise important points. Primarily though, they read to me as polemics of a curiously American nature.
The pattern goes something like this:
- this development is bad
- companies will be unrestrained in their use of this development
- there will be no rules so they can do whatever they want
- we are all fucked as a result
But then...propose that we make some laws to put rules around this stuff, also known as regulations and everybody goes "whoa hold up hold up hold up...I dunno about that part."
Dear friends - America has always been this way. Study your 19th and 20th century history. Companies will exploit the shit out of us unless we put some rules in place to prevent it. Yes, that might mean making less money in the short term as regulations cause friction. But in the long term it means we can have a better and actually livable society.
(For what it's worth I'm an American and not an uppity European or Australian taking potshots from across the pond; no offense to Euros or Aussies intended, love you guys)
I sent the entire series by Aphyr [1] to some friends. Two of them, independently, responded with a variant of, "TLDR, can you give a summary?"
I chat with these friends a lot but I rarely send articles that I suggest they read and that I think are profound, so I expected them to read it. These are smart people that have a history of reading lots of books.
They are both huge AI proponents now and use AI for nearly everything now. Debates on various topics with them used to be rich; now, they're shallow and they just send me AI summaries of points they're clearly just predisposed to. Their attention spans are dwindling.
[1] https://aphyr.com/data/posts/411/the-future-of-everything-is...
Regarding companies trying to block any contact with customer service and adding endless AI hurdles: In some countries, having a reachable means of contact is legally required. Is there a NOYB-style organization that specializes in enforcing this right (suing companies on behalf of consumers)?
For the "bureaucracy has royally fucked up and doesn't want to fix it", if it is something that can be fixed with money and isn't time sensitive (e.g. you need a refund rather than get the airline to actually provide you the ticket you already paid for and want to fly this weekend): In countries that have effective small claims courts, these can be a surprisingly convenient (less hassle than the "talk to the bot" wall of the company!) to resolve this kind of issue.
I hope that these resolution methods become more common - I think the tools to fight enshittification often already exist, we just don't use them enough. A welcome side effect would, of course, be that this would impose a real cost on the enshittifiers, creating an incentive to provide proper support.
> Since LLMs are unpredictable and vulnerable to injection attacks, customer service machines must also have limited power
Haha yes. I interacted with a bank one. It was like press 5 for mortgages but with a text to speech front end.
At the end of the day the LLM can be tricked into doing anything.
D^HLying is easy, it's comedy that's hard...
„Agentic commerce means handing your credit card to a Large Language Model” - this is simply not true. LLMs/Agents will never get any credit/debit card details, they will be just an interface.
So, providing actual customer service becomes a market differentiator?
"Yes, we cost more, but your get what you pay for" can be a good play.
Excellent essay. I see some of this is already happening imo
This is doomerism. Yes, everything will get worse. But everything will also get better. Such is progress. (for every one of these examples of annoyances, I can think of two ways to use AI to get around the annoyance. not clever programmer things, but things an average person who learns to use Codex or Claude Desktop to operate their desktop will know)
Most of these annoyances are also things that existed before AI, and will continue to exist after, because consumerist capitalism. The good little obedient consumers get abused because they don't stand up for themselves. Customer service is an enfuriating maze? Yeah, because you voted with your dollars (and political indifference) to allow companies to make customer service (the thing you pay for) worse. We bring these problems on ourselves. It's pointless to complain if you aren't willing to do anything to change it. (And if you think you can't change it, there's other nations to look at, as well as the fact that you live in a democracy - for now - unlike the rest of the world)
Hell, we already have companies whose sole purpose is to manage your subscriptions for you because you're too lazy to do it yourself. You could look at this and say, man, the world is terrible! Or you could look at this and say, man, how great is my life that I can not only subscribe to a lot of things without going bankrupt, but I have extra cash left over to pay a company to manage my subscriptions?
Don't let the hedonic treadmill and complacency trick you into A) accepting a worse life, or B) convincing yourself your life is bad when it's actually better than most people's.
> ML models will hurt innocent people.
Lots of blaming LLMs but I think the root cause lies elsewhere, I’m not even sure whether dismissing it as “capitalism” or “profit motives” would do it justice, because in general it feels more like the world that we live in lacks humanity.
Even in a capitalist world, a company could take a stance and decide not to purposefully screw people over, but in the world that we live in instead they look for ways to better screw over people and extract more money from them. It doesn’t matter whether your customer support is handled by someone from India, a crappy telephone tree or some voice model, when the incentive is the same - to do the bare minimum for customer “support” (in practice, just getting you to fuck off). Same for handling insurance claims and “dynamic pricing” of things - it doesn’t matter whether it’s some proprietary algorithm or just an LLM making crap up when the goal is to screw you over.
Blaming “AI” for all of this would be barking up the wrong tree (without that tech they’d just find other ways), though one can definitely acknowledge that this technology provides another convenient scapegoat, same as how you can lay employees off and just say cause it’s because of AI when in actuality it’s just greed and wanting to make your books look better.
AI on AI warfare
I read the first couple of posts in the series. The essay is full of criticism of LLMs, and in a couple of places the author distances himself, as if he himself isn't using them ("some people I respect tell me that...").
It's certainly worth discussing the fact that the entire industry is starting to outsource large amounts of our thinking and writing work to non-sentient statistical algorithms, but this discussion needs to honestly confront the extent to which they are successfully completing useful tasks today.
At various previous companies I've worked at product managers, executives, and engineers love bandying about the idea of "building for nontechnical users" as a way to make their widgets more "friendly". But it's just another way to otherize and denigrate "those people" who are the out group. They might, through a metacognitive defect or simple sociopathy, actually believe they're "doing good" by considering the poor creature's plight and making compassionate decisions on their behalf. But it's all crap. All they're actually doing is confirming their biases. LLMs are the divine nectar to these people, an enshittification accelerant par excellence.
Everything that is old is new again.
Payment processing, is better than it was in 2000, but still not good.
Micropayments: this is obnoxiously expensive to do.
Discovery, and discoverability: again here we have better but not good solutions (and many of the ones that were once good are enshitified).
Pricing: this is a problem everywhere, and frankly we need the law to change in a way that is pro consumer. Publishing prices, disclosure of fees, in both services and for payment processing (that 3 percent back from visa looks a lot less attractive when it's part of a 5 percent mark up).
Customer service: well there are already companies promoting models where they cut you off and send you into a black hole (google is a prime example). Good customer service will become a differentiator, and maybe a "paid for" service as well.
[dead]
To lie requires recognition of the truth and an intention to deceive. LLM’s don’t have such abilities. They are systems that generate plausible sequences of symbols based on training inputs, alignments, reinforcement, and inference. These systems don’t know or care what truth is and therefore cannot lie.
It’s already bad. I’m not looking forward to the future. These systems are terrible. It’s a future without people that they want for some reason. I’d rather deal with people incompetent, tired, annoyed people than an LLM.
Don't agree with this.
LLM when it came out, was perfect as an interface between a system and a normal human.
So many people call customer support for issues they could in theory fix themselves. If that LLM system can understand me well enough, its an okay interface.
In worst case you have to escalate anyway. My mum actually told me that she talked to some AI.
And yes normal systems are also not correct often enough. With AI/LLM software will get cheaper which should incresase quality overall.
I dont think ai/llm in this case will change anything.
Relevant change will happen due to the fact that humans can be replaced by AI/LLMs. It was not even imaginable a few years back how a good ai system would even look like. Translaters lost their jobs, basic arists lost their jobs. Small contracts for basic things are gone. The restaurant poster no one cares? AI. The website translation for some small business? no one cares.
So basically more ways of trying to make people buy things, do things, think things than before? I feel like our whole world more and more circulates around manipulation and the absence of truth and discourse.
Then again, I do think LLMs are an incredible technological achievement. The issue is not so much what they do or that they exist, but how they are utilized. Right now, they are utilized to further the class divide between rich and poor.
Who are we to trust in the future? Not big companies, not the state, not LLMs. Time to organize around groups and collectives that we know we can trust and that we know have our wellbeing in mind.