alt Hacker NewsSo, a couple years ago Microsoft was the first large, public-facing software organization to make LLM-assisted coding a big part of their production. If LLM's really delivered 10x productivity improvements, as claimed by some, then we should by now be seeing an explosion of productivity out of Microsoft. It's been a couple years, so if it really helps then we should see it by now.
So, either LLM-assisted coding is not delivering the benefits some thought it would, or Microsoft, despite being an early investor in OpenAI, is not using it much internally on things that really matter to them (like Windows). Either way, I'm not impressed.
Replies
I know blaming everything on LLMs is in vogue right now; but this is much more to do with Microsoft very publically firing the QA department[0][1] as a cost savings measure and claiming developers will do their own QA (long before LLMs were on the scene). It started in 2014 and the trickle never stopped.
Microsoft has a cultural problem; it went from an "engineers" company to an MBA directed one, trying to maximize short-term shareholder value at the cost of long-term company reputation/growth. It is very common and typical of US Corporate culture today, and catastrophic in the long-run.
[0] https://arstechnica.com/information-technology/2014/08/how-m...
[1] https://www.reuters.com/article/business/microsoft-expected-...
I fully believe highly skilled people can get a great benefit from LLM tools; probably not 10x; but enough that its noticeable.
The key thing for me is that it only works when the LLM is used for tasks below the devs skill level; It can speed up somebody good, but it also makes the output of low-skill devs much harder to deal with. The issues are more subtle, the volume is greater, and there is no human reasoning chain to follow when debugging.
So you combine that with a company that has staff in low skill regions, and uses outsourcing, and while there might be some high skill teams that got a speed up, the org is structured in a way that its irrelevant.
Oh it did help.
Microsoft went all in on do more with less and fired/reorged significant part of the company.
Wouldn’t be surprised if the outage is caused by new team taking something over with near zero documentation while all the tribal knowledge was torched away
They weren't great before LLMs either.
Also, it seems from the outside like a dysfunctional organisation, or at least with incentives heavily misaligned with their users. Replace LLMs with a bunch of 10x engineers and it will still be bad in an environment like this.
So not sure how much to blame the LLMs - or in fact how much MS is really using them. Poor souls have to use MS AI tools, I almost feel sorry for them.
this reasoning is flawed.
wouldn't a for-profit company just balance the workforce for the productivity gained to increase overall profit?
some person is 10x 'more productive' (whatever that means) , let's cut 9 jobs.
Although to your grander point, employment during the LLM-embrace period seems fairly stable.[0]
If they used copilot and it was years ago, I'm actually impressed there are no reports of Windows PC's exploding
Imagine a world where Microsoft was pushing “Copilot” integration everywhere, just as they are in this one—but the proof was, actually, in the pudding. Windows was categorically improving, without regression, with each subsequent update. Long-standing frustrations with the operating system experience were gradually being ironed out. Parts of the system that were slow, frustrating, convoluted, or all three, were being thoughtfully redesigned without breaking backwards compatibility, and we were watching this all unfold in real time, in awe of the power of “AI”, eyes wide with hope for the future of software, and computing in general.
Think of how dramatically this hypothetical alternate reality differs from the one we live in, and then consider just how galling it is that these people have the nerve to piss on our leg and then tell us it's raining. Things are not getting better. This supposedly-magical new technology isn't observably improving things where it matters most—rather, it's demonstrably hastening the decline of the baseline day-to-day software that we depend upon.
[dead]
> If LLM's really delivered 10x productivity improvements, as claimed by some, then we should by now be seeing an explosion of productivity out of Microsoft. It's been a couple years, so if it really helps then we should see it by now.
That productivity may not be visible. I think MS's move-everything-to-rust initiate would be one hell of an endorsement if they manage to make visible progress on that in the next couple of years.
Microsoft is not even using dotnet core and what not, internally. SLT is very hard on adopting AI, but not much on getting results