alt Hacker News> A study from METR found that when developers used AI tools, they estimated that they were working 20% faster, yet in reality they worked 19% slower. That is nearly a 40% difference between perceived and actual times!
It’s not. It’s either 33% slower than perceived or perception overestimates speed by 50%. I don’t know how to trust the author if stuff like this is wrong.
Replies
piker • yesterday at 11:01 PM
I get caught up personally in this math as well. Is a charitable interpretation of the throwaway line that they were off by that many “percentage points”?
➕ show 1 reply
regular_trash • yesterday at 11:01 PM
Can you elaborate? This seems like a simple mistake if they are incorrect, I'm not sure where 33% or 50% come from here.
➕ show 2 replies
softwaredoug • yesterday at 11:03 PM
Isn't the study a year old by now? Things have evolved very quickly in the last few months.
➕ show 1 reply
> I don’t know how to trust the author if stuff like this is wrong.
She's not wrong.
A good way to do this calculation is with the log-ratio, a centered measure of proportional difference. It's symmetric, and widely used in economics and statistics for exactly this reason. I.e:
ln(1.2/0.81) = ln(1.2)-ln(0.81) ≈ 0.393
That's nearly 40%, as the post says.