alt Hacker NewsThe end of Microsoft Research's AI Frontiers group
Comments
This refers specifically to "AI Frontiers", a research group within Microsoft Research that focuses on fundamental advances in AI. There is no end in sight to real AI frontiers.
Perhaps "The End of Microsoft Research's AI Frontiers Group" would be a less ambiguous title.
Companies with billions of $, stopping R&D on the most revolutionizing technology/ breakthrough is something I can’t understand. I do understand there’s an AI Race to win, and that is with transformer and current LLMs architectures, but if research could find a better, more efficient approach they would re-shape this race, lead it, or even dominate it.
To me, all signs point to US over investment in hardware capacity to support the "frontier" when in reality the next major breakthroughs are likely radical optimizations that end in increased consumer training and inference efficiency.
I'm not sure how that, combined with open models inching up on frontier intelligence, is compatible with these massive AI IPO valuations?
It seems like closed model AI demand has a lot of contingencies that are ultimately fragile and possibly short sighted.
He didn't say they were not doing transformers research. In fact, he specifically mentioned a type of transformer they were working on. And labs like Qwen and DeepSeek (at least) have shown how incredible results can be when you innovate in the area of LLMs.
I think the real point of the post is that office politics and poor management got in the way of innovation due to infighting and poor execution in transfer of technology from research to product.
Almost as fascinating is that the article references Digg as the source - genuinely hadn't heard of Digg for maybe a decade? I thought the DVD encryption fiasco + reddit had ended it?
John was the creator of Vowpal Wabbit, which used to be a popular ML library: https://en.wikipedia.org/wiki/Vowpal_Wabbit
It was my introduction to feature hashing.
The headline makes good bait for AI skeptics. But this is a team of 50 people with a grandiose name.
The more interesting idea is that MS has abandoned AI research entirely, presumably because their zillions invested in other companies are a better bet in that area.
This seems to track with a recent blog post from MAI regarding their approach to pre-training their first Frontier level model which seems promising but does seem to have a focus on scaling.
https://microsoft.ai/news/frontier-pre-training-at-microsoft...
Is this the same group that did ResNet? Bad news
The Microsoft AI (MAI) part stood out to me. I bet the politics of being in Mustafa's group were not fun.
Lack of focus and long term planning at MS
Not what i expected, but good read nonetheless
If you value the freedom to pursue your own research direction and share your work freely, then choose academia. All the perks of working in an industry research role are provided in exchange for this freedom.
If your research aligns with the incentives of industry, you can get the best of both worlds in industry. This was the case for machine learning researchers in the post-AlexNet, pre-ChatGPT era. The good times can't last forever, though.
It seems as if Microsoft is losing developers now.
The whole "let's go into AI" is not really compatible with oldschool software jobs. AI drains humans dry.
I mean we are already seeing the hype cycle coming to a close, with the stock market showing this, but key people leaving kind of means that Microsoft is losing its profile here. Add to this that many don't like Win11 and you see a huge paradigm shift here - and tons of associated problems.
"In practice, there are substantial issues because the philosophy of the ex-Inflection leadership of MAI is incompatible."
AI is killing people here - not necessarily literally (though we have to see how much economic damage AI has caused) but figuratively speaking. The counter-wave is coming here.
Executives chasing short-term returns. Google did 20 years of AI research, even creating custom chips, just so we could get to this point. Without long-term strategy Microsoft will perpetually be the also-ran. (which I guess is fine with shareholders?)
(In the context of Microsoft Research) Man, the history of how large corps fumble major wins should be studied.
I remember during my thesis how Microsoft Academic (a Google scholar/Scopus alternative) was ridiculously good at its job.
I'm talking, "input the key words and filter out the wrong fields and you've got 20 highly relevant papers that no other search term has found" level.
...Of course, they proceeded to close it down.
I can't even express in words how mad and disappointed I was and still am at Microsoft for this alone. It felt the closest to a genie or an oracle in being amazing. (The old Swype vs modern Gboard is probably a close example for those who'd used both.) It's just... crazy, frustrating, and crazy frustrating.
That's a name I haven't heard in a while
Some more discussion on source: https://news.ycombinator.com/item?id=50406589
Mr. S Altman just said almost all Startups will die within a couple years. This is sad.
Well I never thought I’d see Digg used as a primary source
This is certainly an interesting timeline in this section of the light cone
Yay China!
Real research at most big companies is dead. Unfortunately, what has replaced it is mostly venture-backed startups, where the timeline to a return is still expected to be quite short.
As a result, you end up with companies doing basic research under the guise of developing a product, announcing timelines which are never going to be met. Fusion energy and quantum computing startups are some of the most egregious examples of this.
Microsoft is a really shitty place to work right now. There are a hundred stories like this all over the company like this.
[flagged]
His complaints sound childish to me. If the research products were truly innovative and useful, he wouldn't need to force Microsoft to adopt them.
[flagged]
This is true in every large company right now.
The transformer based approach is seen as a highway to race down in terms of training and building better models with minimal organizational distractions allowed while every company puts its foot down on this highway. Exploratory research into other AI methods is highly discouraged for now and organizations are encouraging transformer based scale out and iteration as a top priority.
As an example Meta literally fired it's previously head of AI who held views that transformers won't scale. He wanted to veer off the highway and heavily fund exploratory new types of models. They got someone willing to scale the current method out. Essentially the messaging it "Foot down, there's a straight highway ahead don't divert".
I have the view this makes sense too. Current top models are mere (mere in big datacenter terms) trillions of parameters right now. We definitely haven't hit the ceiling and everyone's noticing "Hey this Opus 5.5 with the increased parameters seems much better than the last one". Well that's a trillion parameter model. We'll soon start hitting 10's of trillions and 100's of trillions of parameter models. Unless there's a ceiling which nobody expects they should keep scaling just as they have been. I really do agree that it'd be insane to veer off this highway with exploratory research at this particular point in time. Foot down, get to 10 trillion parameters, see if the scaling holds, if so off to 100 trillion parameters. If the scaling continues to hold we'll be asking the AI to do the exploratory research for us soon enough.