Ah, it’s a good time to check in with gwern on our conversation about oAI vs Anthropic: https://news.ycombinator.com/item?id=40816755 and our predictions (ca two years ago).

Upshot - poetry expertise does not seem to be the primary focus these days, perhaps to the detriment of the entire world. We did move on from training scaling to “test time” scaling (which I hate as a name btw), Ilya does not seem to have been needed, (although I am really curious what he’s building).

My prediction that you want to be deeply embedded and really rich and part of global infrastructure feels good. My suggestion that oAI / MS would be able to use the lead in 2024 to extend was wrong.

Neither of us talked much about coding as a product that would drive value and behavior, which is super interesting to me, we were probably six months from seeing real competence of any sort there way back in June 2024.

We both seemed to think there would be a single breakout company, or could be one, (although I did suggest buying the basket), clearly not the case with GOOG oAI and Anthropic all posting serious revenues this last quarter / year.

One area of Anthropic that was nascent in 2024, but that I have come to think is super valuable is their mechinterp group. I still don’t see work done by other labs (at least published) to nearly the quality of Anthropic. And the group has clearly moved into a period of productivity; there’s a good chance in my mind it could provide a truly enduring strategic advantage as a tool to be used by the taste makers steering the ship. In 2024, interpretability seemed almost impossible to get a handle on — today, the sustained chipping away at the problem makes a lot more look possible.