alt Hacker NewsJust as a counterpoint ML and AI research has become much more reproducible over time. I feel like this is relevant because ML / AI researchers are huge power users of AI tools.
Between 2016 and 2021 the share of ML/ robotics/ AI researchers being reproducible (ie contianing code and similar instructions to reproduce) doubled [1].
The major US labs have gone largely closed source (I.e. they no longer publish frontier research) but the Chinese ecosystem has incredibly reproducible code.
This is field dependent obviously but I think it atleast gives reason to be optimistic.
Yes people will churn out fake slop research, but it feels like that can be categorized and then ignored.
That's good to hear about ML and AI research, but most research is not based on computers and so would require laboratory setups to reproduce. Not only is trying to reproduce such findings (beyond what is effectively a sanity check) through simulations a lost cause, if AI can reproduce such research it would be capable of doing such research itself... in which case it would be far more fruitful to use AI to do further research.