alt Hacker NewsI think I have LLM burnout
Comments
The reason I'm getting LLM burnout is from dealing with the obvious neutering and opaque downgrading of all the top models.
Prior to the last 12mos AI companies were hell bent on squeezing out the best results from mediocre models.
But... now that the top models have progressed, those same AI companies have switched their efforts into reducing the computation (cost of a producing a result) as much as possible without being too obvious.
What was an exponential slope in the quality of results over the last 36 months has now nearly flat lined.
Addendum: IMHO results have 'flat lined' not because the models aren't much more capable than a year ago, but because conserving the enormous processing cost (of an over subscribed user base) supersedes the goal of following the user's explicit instructions (e.g. especially if that means more processing cost) to generate the best results.
I've started feeling slightly physically ill when I read Opus output for hours straight. This article rings very true for me. I've started complaining about it with my team; at least have a personal style guide in your agent rules that eliminates emdashes, the "it's not X, it's Y"s, the long lists of modifiers before the noun, using the word "land" to mean finish, etc. I hope this is just a phase of adolescent LLMs.
From my experience, there are mainly 3 burnout reasons. 1. Multi-tasking is the top one. I usually have to frequently switch between 3 to 5 agent windows which are on different things. It's extremely exhausting when each round takes a few minutes. Before coding agent era, I believe most developers had chance to spend 2+ hours focusing on one thing. Now coding agents have increased my spectrum on the tech stack, but the bandwidth to do deep work isn't increased. 2. Agents are good at getting things running without crash, but do not guarantee to produce correct code. This is quite different from human experts with fundamental knowledge. 3. I also get frustrated when reviewing piles of AI generated low quality PRs. My attention is a limited resource. I don't waste too much energy on other people's work, but if I don't spend more effort, the entire project is corrupted quickly by reckless AI generated code without human author's careful thoughts and designs. Working with people who have less due diligence in mind is painful, working with them in coding agent era is 10x painful because they produce 10x shit. It's a team culture challenge that cannot be easily enforced.
I do not have the burnout but I certainly operate similarly to the author. I continue to be unable to establish a workflow where allowing the LLM to generate code that I review is faster than writing the code myself. Literally the only two ways out of this dilemma is to blindly trust what was generated or to generate an uncharacteristically exhaustive suite of unit tests to validate every possible scenario. I just write the business logic myself and have the LLM do a lot of the rest. Boilerplate falls into the latter as well.
We are all going to discover the Ironies of Automation.
https://en.wikipedia.org/wiki/Ironies_of_Automation
https://ckrybus.com/static/papers/Bainbridge_1983_Automatica...
It sounds kind of like being stuck working with coworkers who--while not overtly hostile--need constant hand-holding and repeat the same kinds of mistakes every day and can't even be genuinely sorry about it.
Just because we work with computers doesn't mean we don't take, er, social-damage. Or perhaps parasocial damage, in this case.
This is legitimately the reason I'm looking to leave programming.
I got into programming because the problems of programming were interesting to me. But if the problems go from "figure out why this calculator is off by one in France" to "Get this LLM to stop spamming cutsey emojis", then maybe it's time for a career change.
I’m currently doing a project with someone who only uses LLMs, and it’s exhausting and mentally draining.
Whenever I give feedback on something, the answer is just “let me tell Claude”. The person has no understanding of how everything works, and most of the code reflects that.
The other day he hardcoded in a demo mode, simply because he didn’t even know how to set up a local environment and set environment variables. I’m confused as to why Claude didn’t even knew this, but it might just be the prompting.
I limit LLM usage myself, and if I do use it, I try to use it on extremely specific tasks. It’s the only way it works for me.
I honestly don’t understand how all these companies are getting away with generating AI code. Even in a small project I quickly fall behind on my understanding of the project.
I get burnout from frustration when the LLM just can't follow instructions.
Like when I'm trying to get it to create an image, and the first pass is beautiful, but ten different request to modify it, with different phrasing and even example images, produce the same image ten times. Or when you tell it not to use a cheap hack in AGENTS.md about six different ways and in your prompt, and it still does it again, and again.
It's like arguing with an idiot. And THAT gives me burnout.
Also: I've never once seen an emoji in LLM output. What are people talking about?
is there any evidence that Alec Scollon, the first time blog author responsible for this post, even exists? look up the name. boo this post and the premise behind it.
It's amazing how many do not have experience with the full pre and post SDLC.
LLM burnout is when you catch yourself asking ChatGPT 'how are you' and genuinely waiting for an answer that isn't 'I'm an AI language model'
It's burnt me out too. I'm generating 10x more features and multitasking across 4 disparate projects. My greatest concern is I don't really have a strong connection to the underlying fundamentals anymore. I need to see how the things works to internalize it. Now I just trust that the agent wrote this piece correctly.
The productivity drive and the sheer feature set you can generate in record time makes it easy to forget proper sdlc hygiene.
> My job has changed from designing and writing code to designing code, describing the design to an LLM, reviewing code the LLM produces
As a long-time engineering manager, PM and, eventually, product owner my response is, "Congrats! You've just been promoted to management." :-)
As a new manager, your first challenge will be successfully delivering commercial results using only a team of 'differently abled' new grad interns. Don't complain, new managers don't get to pick their first team! To be honest, these guys are more like alien brains raised in a vat with no direct senses. They've only ever experienced a data feed of the internet and, oh yeah, they get near-total amnesia a few times a day (but maybe you can teach them to write notes for themselves). They also have ADHD and are somewhere on the spectrum. But don't worry because what they lack in common sense, experience and intuition is offset by having a sort-of photographic memory and a willingness to grind on a problem 24/7. You should be fine. Good luck, we're all counting you...
Burn out? This guy is weakling with zero experience in full QA and validation.
I don’t understand what could possibly need to be made so fast that isn’t totally made up billable hours. Running at top speed long enough to be burned out is either ineffective, or valuable enough that someone else can take over while you sleep.
What to do to achieve this kind of burn out, I feel like I am a stubborn old developer, I'm coding since 2013 and I am 25 years old.
My mind still can't function well without having knowledge about everything.
> My main project right now is to establish a framework for large-scale, unsupervised code generation in our codebase
Anyone else working on something like this or know of any projects attempting it?
I think having style guidance in your context is valuable for avoiding this kind of thing. Having to read awful, cliched text all day is even worse than having to read reams of useless code. I have some simple humanizing content in there that specifically calls out the rhetorical devices that AI loves, and it drastically improves the diffs and comments. It also makes the coding performance generally slightly worse, but ergonomics uber alles.
Current AI is like the film company producing TV series or movies
Review AI code line by line is like watch movies frame by frame, and is impossible, very difficult, terribly boring, or abandoned sooner or later.
If an LLM was a person, it would be known as a "bullshit artist". I've worked with bullshit artists (people) before, and its exhausting trying to separate fact from fiction. It's a reason to avoid working with such people.
A good coworker will admit not knowing something, or if unsure give their best guess but discuss its limitations and why they might be wrong.
Question: Has anyone experimented with using voice to directly prompt an LLM, without doing speech-to-text? If an LLM can pick up on the skeptical nuances in a person's response, it might be prompted not to be overconfident in its subsequent output.
split jobs and managing people is hard.
Do does managnig agents.
I feel you. 'But we need to adapt or quickly become invaluable'. It is a tough reality to swallow - me included.
I don't have much success with using the LLM to make changes to a big legacy codebase. Instead, I use the LLM to gripe about things I don't like in the code. Usually, it is a brilliant commiserator.
Avoid Gemini and the lesser ChatGPT models and your emoji problem goes away.
If you can afford it, I highly recommend quitting and going independent at least for a while. It’s so much fun building right now.
I don't mind interacting with LLMs myself and find they increase my productivity a decent amount. I just can't stand dealing with other people's slop.
Getting sent IM responses that are copy pasted LLM nonsense. Getting a massive PR to review that was generated overnight and the author didn't read it first.
Also known as clanker fatigue
I don't really understand how this isn't a self-inflicted problem? Perhaps it's because I'm not really mandated to use LLMs in a particular way, but I've had great success doing a combination of writing code myself and using smaller but faster models as a sort of "flood fill". The larger models can also be useful when you're implementing something which already exists in similar form in the codebase, because you can just put that code in the context and you'll get something very similar outputted. So the more code you write, the better the LLM can be later on. Codebases should get easier to add to the bigger they get, not harder.
Of course if you're supposed to achieve so much output that it's not possible to do anything but vibe it, fair enough.
i think we're interacting with a character not a person.
Tell the llm to answer like a cavemen, if llm talk like cavemen, the answers become shorter and more compressed.
https://github.com/JuliusBrussee/caveman
It's for getting it to output shorter answers, but also could help with your burnout.
started to dread reading LLM output because I know what I’m going to find. False assumptions and hallucinations. Emphatic, staccato fragments. Excessive emojis .
I do not understand these complaints. Yes, those are the defaults and they're annoying, although the general public seems to like them. But you are not stuck with these. You can just tell the LLM how it should interact with you. If you're using any sort of harness beyond the chat window in a web browser, you can codify these instructions in a rules.md file or similar and have it automatically included in any new chat. It's not any harder than changing the default wallpaper or color scheme on your desktop operating system.
In reverse order, you can just tell the LLM to never use emojis. I don't like emphatic staccato fragments either, so I tell it to eschew the language of marketing and hype and stick to a factual and plain language, or to employ an academic tone. I explicitly instruct mine to ask clarifying questions whenever context is ambiguous and to push back on false assumptions or common misconceptions (by me). Hallucinationsa re the biggest problem of those you mention; it's not easy to totally eliminate them (for the same reason it's not easy to instruct people to not fall for scams or disinformation), but you can considerably reduce them by setting standards for citations.
I have ideas about reducing hallucinations over work material (ie a codebase) but am omitting them here as they are not fully thought out or tested.
I had this too. Started after lay off in October.
What helped was a sleep and work system, oriented around being offline that was inspired by nature and from my earlier days in working in tech while car camping across the national parks.
Basically: the sun wins in terms of how all energy on the earth is structured, and expressed. All manners of cycles of organisms and living systems are in relation to its rise and fall, and even its particular color spectrum phases (whether thats night oriented or day). I call this our real circadian rhythm; it's used to being signaled by the light of the sun and maybe fire for millions of years and it isn't until recent centuries when we started tricking our biology with LEDs and lights. So the solution is simple. Orient yourself around the light of the sun and make sure it's the first and last major light source you see; blue limiting is the most important part BEFORE sunrise and AFTER CUT OFF ALL BLUE LIGHT. On my Mac I use a red light filter (using it now, it's 11:07pm ET and the sun went down about 2.5 hours ago). It's really hard to stay alert and chatting with an LLM when the only light sources are red and you keep them dim at that. Our ancestors would rest when the sun's at its peak (~1:05 pm today) and that's a good time to divide my own day productively as well. With intentional breaks diving the middle of the day with sunlight anchoring it, my nervous system is more relaxed, and by the evening time, it's also ready to transition out of anything blue-light assisted and most intellectual work and problem solving falls into this bucket. It's really hard to explain but it really works so simply. To enjoy the process a little more I made this fun sun clock, check it out at https://sunsignal.app
LLMs poison your mind. The more AI slop you read, the more your mind turns into something like slop. This isn't very different from the idea that the food you eat is what your body is made of.
[flagged]
[flagged]
[flagged]
I don't think I have a "burnout", but LLMs are really exhausting due to amount of pressure they generate. No one is really pushing me to increase my workload, but at every moment there is always something ready, done by my clankers or clankers of other people that I could be unblocking. In the past (before LLMs) it was already hard to keep up, but now it feels like there's 10x more things waiting at any given time, and there could be 10x more if everyone just "optimized" and streamlined processes fed the AI even more tasks in parallel faster. It just being a bottleneck of everything, all the time is tiring...
I am happy about all the little side-projects, and ideas it help my realize, and I enjoy exploring this new world, but I've noticed LLMs feed my unhealthy "don't want to take a break and waste time being idle" mindset, and I need to correct it.
W.r.t. article's main complain - I think the similar thing happened due to factory manufacturing automation. What used to be a varied skillful craft in a shop became standing in a single place of an assembly line doing the exact same thing whole day. LLM took away the more creative and variable part of the work, and left the repetitive QA rubber-stamping. Probably some of the mitigations used back then could be rediscovered today.