alt Hacker NewsIf not ”missing the forest for the trees” is a requirement for debugging an ML pipeline then that offers a path to building such a large machine-verifiable dataset, does it not? That’s what labs are currently doing as I understand it.
If not ”missing the forest for the trees” is a requirement for debugging an ML pipeline then that offers a path to building such a large machine-verifiable dataset, does it not? That’s what labs are currently doing as I understand it.
As someone who doesn't work in ML, I can only guess what drives the recent improvements.
This kind of post-training and improvements to the training pipeline is exactly what my guess would be.
Curated tasks where the efficient approach is incentivized over getting bogged down in details. This behavior should then generalize.
It seems like a much more likely explanation than the fantasies about illegally training on user data that I often read on HN.