alt Hacker NewsI think a lot of us are trying really hard here to avoid the simplest and most obvious answer, which is that AI coding software is helpful, but doesn't deliver the kind of productivity gains boosters are talking about.
There's a bunch of reasons for this, but the obvious one to me is just that I haven't seen any backed up claims that actually break down how a 10x (or similar) speed up would get delivered in practice.
For the sake of an example, let's imagine an engineer who works an 8 hour day split up like this:
- 2 hours: investigating and debugging potential issues
- 1 hour: reviewing code
- 2 hours: in various team meetings, catch ups or otherwise
- 1 hour: designing (including at a level of writing code for prototyping etc)
- 2 hours: actually writing code
I think that's a semi-reasonable day breakdown. Realistically, I think AI helps debug faster, and generate code faster. Possibly design work too, but I think this isn't really the case since AI workflows necessitate a lot more design work in order to make the AI generation deterministic.
So looking at that day, if I was able to fully automate writing code and debugging issues so that it took 0 time at all (which I think is a very optimistic assessment of the actual speed ups AI provides), we're looking at a 2x speed up of an engineers productivity.
If we think AI makes those things twice as fast (so 2 hours, instead of 4) the engineer is now at a 1.25x speed up. That's great, but it's hardly revolutionary. It'll change your output something like the way going from a team of 4 to a team of 5 might do.
This is obviously complete conjecture and I'm sure the average time engineers spend on things is different, but I think there's an obvious take away. If you want to make claims about the performance impact of code, you need to know where the code spends its time, and what impact you're making on those sections of code.
Similarly, if you want to know what performance gains are being made, you need to know what engineers spend their time doing and what impact AI is having of those tasks.
I've seen very little attempt to actually make realistic claims on that basis, and a lot of vibes-based "wooah, AI makes me so much faster" type claims. Those claims are obviously not nothing, but I wouldn't trust people's gut reactions to make serious claims about the overall productivity changes in our entire industry.
Replies
You've also skipped the 1 hour reviewing code becoming 2-3 hours, or even more.
I think there's a type of company for whom this works; a small team really can deliver a massive volume, but it cannot be constrained by your typical software bigco superstructure. Ops/Readiness/Process Engineering/Enterprise Arch/Risk etc, entrenched concerns with soft power through pure delivery friction.
You need to be an an org who is able to deliver directly to your consumer, who needs to be defect-tolerant with the expectation of quick iteration. That customer will have access to advanced tools that its competition is unwilling to adopt. Many will fail.
There are many verticals for which this does not work at all - but this is unlikely to be the permanent state. The new model will work its way through the market, and companies will fall as fast-moving competitors find a willing customer segment they can target, make that segment out-compete the others using their access to tools with superior abilities but higher defect rates. The customer base will eventually adopt, having become accustomed to this, and so slowly expectations will change, and so will our field.
I for one welcome our slop overlords.