It does provide a useful ceiling, at least. If even Anthropic is seeing only a 2-4x revenue multiplier from AI adoption, I’m intensely skeptical of the people who run around saying they produce 10x or 100x more valuable output because they’re really good at building a software factory or whatever.
You're missing the obvious. Time. There are highly talented developers who have the technical ability but had no time in the past to do these things because of real life obligations besides their job. Many are producing 10x more valuable output now. Also you're missing that Anthropic as a leading frontier lab isn't a good comparison for almost any metric for any other company in the world. It defies any principles companies usually have like unlimited talent supply.
There is effectively no value in statements like these.
There is no measurement. Think about how much of a difference empirical science made in such a short amount of time. Or think about how it is impossible to optimize software without measurement etc.
Also there is the AI companies’ atrocious reliability even if you trust the measurements
I mean, your productivity gains are relative to your starting point in some way.
If you are already an advanced company filled with skilled people, you may not see the gains that a mid-tier company filled with mediocre seat warmers would see when their "best and brightest" start adopting AI to replace said seat warmers.
It's easy for AI to make somebody 100x more productive if they were insanely unproductive in the first place. In the workplace this is unlikely, but in hobby spaces I believe it. Lots of hobbyists using AI to program micros/etc that, in the past, would have been beyond their capabilities without extensive study first.
Consider for instance the bicycle youtuber Berm Peak, who's been using AI to reverse engineer and hack shitty ebikes. This kind of work is outside his expertise, so the relative lift he gets is massive.