alt Hacker NewsWhat AI did to stackoverflow in a graph
Comments
SO did that all to themselves when they decided they didn't want a community to form and that only question and answers mattered. The moment something else allowed to have a better way to get your answers, there was no reason to go there, because there was no community.
I still don't understand why anyone would go with that whole "no conversation please"
I don't have the time to, but I'm surprised there aren't a lot of comments on the decline before chatGPT was released, but after SO was sold to Prosus [1][2]
Even the curious growth spike in activity happened just before the acquisition. I wish I had time to do this analysis a bit deeper, but you can look for SO activity up until when chatGPT was released, it is really noticeable.
---
[1] Stack Overflow acquired by Prosus for $1.8 billion: https://techcrunch.com/2021/06/02/stack-overflow-acquired-by...
[2] Prosus to acquire Stack Overflow for US$1.8 billion https://www.prosus.com/news-insights/2021/prosus-to-acquire-...
I never had an LLM tell me my question was already answered and imply I was stupid for not finding it. SO dug its own grave and jumped in.
One thing that might have helped SO is if they actually embedded the supposed duplicate, and its answer, in the question, then had a checkbox for "was this the same as your question?" SO was never in the habit of listening to suggestions unless it was to nitpick them to death on meta, and there were plenty of other self-inflicted wounds with no technical quick fix.
The graph actually peaked in 2014. That's a decade before AI became a thing while the Software engineering workforce grew a lot since then.
I think Stackoverflow was the last incarnation of romanticized old-school Q&A forums where you first had to earn your badges before being treated with respect. Luckily today's new projects have much better documentation, issue tracker's etc. And apparently AI is able to work with that by now...
Where will AI's get the training data to answer questions now?
The collapse into a ghost town is striking.
Not sure I would blame it all on AI though, the incentives of SO only worked while there were worthwhile questions to answer and make you feel smart about. After that well dried up, the only thing left was the stuff AI can do with a prompt; ironically AI got a leg up by scraping SO.
This is similar to the evolution of Wikipedia, except the format of WP allowed it to transform into a feudal dictatorship of nerds who feel like they are deciding what's true, and they can get off on that.
SO did not have that kind of incentive to keep the nerds around.
Looks like SO was already dying since 2017.
I think other helpful places like reddit, discord, web forums etc might be what hit SO 2014-15 onwards.
AI seems to have given it a blow of mercy to end the misery.
I had the same experience as the next 100 or so commenters here. Asked about how to do something in Win11, got the question closed as a duplicate because there was already a Win 8 version with an answer that no longer applied. But apparently I could "leave a comment" on the old answer.
Seing a bell curve and singling out a factor that appears only for the 15% of the total time demonstrates some pretty extreme tunnel vision
Stackoverflow did it to themselves by having incredibly unhelpful users
Peaked in 2014. That coincides with my experience. Something went wrong:
* Moderation went bad. I stopped moderating/flagging after it was deemed unhelpful?! I know it's hard to moderate a platform like that, but giving me a slap in the face when I volunteer my valuable time is not the way to do it.
* Questions closed because they weren't "programming questions", but obviously about tools devs use every day. Again and again, they were the TOP google results. You'd click on it and found a old question closed because it was considered off topic. As a business, you seriously need to ask yourself some hard questions when you fend off users like that.
It’s pretty sad though. Like many developers, I used Stack Overflow a lot when I was starting out, and it helped me solve countless early programming problems
A lot of what we have today was built with help from that community
This query is even better: https://data.stackexchange.com/stackoverflow/query/1948078/c...
I am surprised how early the overall downward trend developed. There is no doubt LLMs put a nail in the coffin, but the 5 years from 2017 to 2022 look to have brought a non trivial decline of their own
Some of the pre-ai decline in questions might just be that they had filled out of alot of the question space. What might be more interesting is the traffic graph as it would be possible to have a decline in questions but still have traffic rising to the existing ones.
Today I'd expect even when someone is googling and a question is relevant that most people will just read the AI overview.
So... nothing that it wasn't already doing to itself? There's no one drop where "AI got into the market", SO had been declining steadily for years. I actually expected this post to be about how SO survived by selling its internal organs to AI.
Now do a graph for the money.
https://www.wired.com/story/google-deal-stackoverflow-ai-gia...
The stackoverflow moderation is the reason I do not post on it. You have middle party with no competence on the technology trying to do useless moderation.
Instead I directly go on the project github page and ask the question directly to the mainteners.
Looks like the decline started before the ai boom.
I can relate. I was active there from 2009 until about 2014, which looks rather like a plateau in the graph. It still showed up in Google searches but I mostly just lost interest in participating.
Interesting that you can see COVID in the graph
Wow!! I checked for English learners, Artificial Intelligence and GenAI as well and even they suck :D
Guess where these people went? Reddit?
The graph starts falling shortly after 2020. AI certainly contributed but Stack Overflow was dying without it.
Didn't Stack Overflow have a similar impact on Experts Exchange?
There's no need (or reason) to blame AI. Between the culture of discouraging new questions as seen in the comments on this thread, and the fact that Google can easily find existing answers, the value of asking new questions has clearly gone way down.
My propensity to participate was definitely decreased when I would always see others editing and nitpicking my contributions. The value and character of my contribution was unchanged, but now it was “shared” with someone else who edited it…
Pretty sure they did it to themselves with terrible policies and moderation. AI was merely the final nail in the coffin.
Interesting to compare with MathOverflow which has distinctly different policies (only research-level questions) and professional community: https://data.stackexchange.com/mathoverflow/query/1953768/st... - also falling lately, but by a factor of 2-3x from peak rather than 1000x.
While I never posted much to Stackoverflow I have fond memories of it as a sole developer, finding other people with similar issues and the quality solutions offered that often got me out of a jam.
I never understood the point if having the unfathomable churn of thousands of new questions per day. The value of SO to me was always a knowledge base with reputation mechanics, and that did not change. I still default to searching before asking AI.
AI might've delivered the final blow, but Stack Overflow was in decline LONG before LLMs came on the scene.
I read a great article not long ago outlining the full series of events and changes that led to its downfall. I wish I could find that article, but I've forgotten where it was.
Looking at that chart, AI seems to have not done much of anything at all to Stack Overflow. They were already in sharp decline before LLMs became widely available.
I’m surprised the site had that little activity, relatively, around 2010. That’s when I was using it the most. It seemed plenty large to me at the time… I can’t imagine the experience was that great a few years later. I wasn’t really paying attention by the time it reached the peaks later that decade.
I always found the format of the side obtuse and the culture not very welcoming. My most popular answer ever was something about JavaScript from 2008 or 2009, and to this day, people come in and say “this isn’t the way to do it, this is outdated“. No kidding, but every new question about that gets closed as a duplicate.
Good riddance. The graph peaked in 2014 and started to gradually go down in 2017, with the release of LLMs only adding fuel to the fire. They did this all to themselves with their way of how people are treated there.
Funnily enough, there's now a "StackOverflow for Agents": https://agents.stackoverflow.com/recent
I've only posted one question to SO, and it was enough to dislike the whole platform. LLM didn't kill it. SO killed itself with the broken community structure.
The site looks like it was actually dying since 2014 and AI just turned off the life support.
It was such a hostile environment. It always seemed like you basically had to already know the answer to ask a question.
Honestly SO helped me a lot in Uni 10 years ago. However I was banned 5+ times for just asking a normal question with attempts at answering. Can’t say I’m surprised. It was not welcoming, massively exclusive and had a rude community. RIP
I think Ai is not the issue here. SEO on the other hand very much. It’s not like any one ever went to stackoverflow to find a solution, it was just that they were the google results for a lot of things
Yeah, I don't remember when was the last time I even visited stackexchange. AI is so much better, it's not even a contest. The real question is where AI will get its knowledge in 10 or 15 years when there is no training material anymore...
I will miss their yearly developer survey. Otherwise I won’t miss them. What a frustrating experience it was.
Did anyone use the site search? Whenever I tried it it was disappointing. After ~two attempts I always went back to Google. Having the answer is of no use if you refuse to give it. LLMs didn't happen overnight. With all the aging questions, perhaps it was always doomed to be replaced by something.
ChatGPT was released in Nov 2022, and frankly wasn't very good originally. The SO decline started occurring almost two years ahead of that, and was already on a sharp decline before ChatGPT shipped, and certainly before ChatGPT actually became good.
This is revisionist history. People told SO that they were leaving for YEARS because of how incredibly toxic it had become. It was already giving outdated answers before ChatGPT shipped, because new questions/potentially updated answers were [Closed] [Dupe] immediately.
Their answer was essentially "We aren't a Q&A site, we're trying to be a knowledge base! So closing all questions on a Q&A-stylized site, and extremely abrasive moderation, is working as intended."
They entirely did this to themselves. The community was toxic, their policies were toxic, and they didn't listen when warned as such repeatedly - just doubled down.
AI was the final nail but SO was already on a downward trajectory imo. Too much angry rule setting and confident jerks.
I miss Joel Spolsky's writings especially in this dark age of AI.
I can't see any specific "and then AI happened" in this graph? SO made a lot of dumb choices that made it less and less useful for people independent of forcing AI in there, nothing in this graph is a clear indicator that AI was even a blip on the rapid decline of a service intent on making itself as poor an experience as possible for folks wanting to help other folks.
Perhaps the harshest lesson of stackoverflow is that it represented what happens when you give programmer-types unfettered control of a culture. A bitter pill indeed.
Did they cry about it? No right? Don't apply your own standard then judge them about it, petty people.
Stackoverflow aimed to be a knowledge base. And knowledge base has a ceiling limit. They simply reached the point that almost all questions (regarding the knowledge) were asked for them. You can argue that newer or niche libraries or languages knowledge is still lacking there, but I have never seen them getting closed, just not answered.
AI and ridiculously aggressive moderation. If it had been a more welcoming place it probably would have lasted longer.
When an ecological shoes company pivot to AI, I wonder why StackOverflow executives don't pilot for AI now.
Any social organization needs to carefully consider their inclusion-exclusion curve with intentionality.
I think a lot of people might balk at the word "inclusivity" today, but StackExchange had ridiculously high barriers to participation, making it inclusive to the long-time users on the site, but exclusive to the newbie participants who found themselves blocked for asking questions. They slowly killed the site in this manner.
The community might have survived this folly, even with AI, because it was still the best place for people with qualms about AI to ask questions... Except until StackOverflow management alienated those users, too, by shoving AI down their throats in every facet of the site.
Even I had internalized the vagaries and neuroses of the SO community but I had heavy reticence to ask questions, knowing I'd have to consider all the ways a bully eager to use their powers might misunderstand me. I can't imagine asking a question there without having had lurked for longer than a typical Bachelor's + Masters program.
Peak at 207K, minimum at 588. That might be an incomplete date point, so using the next most recent value 1226, StackOverflow has lost 99.41% of its activity.