Building an AI-native engineering org

> None of this was free. Through the summer, much of our engineering effort went into tests and tooling, and customer-facing changes per engineer stayed between 17 and 30 a month

This is the reality I think for a lot of orgs and is absorbing a lot of the productivity gains. The gap between vibe coding a green field app from nothing which is inherently "AI native" and implementing AI tooling into complex orgs with large amounts of legacy infrastructure, integrations, ingrained practices etc is enormous. For us, it has been a big effort to enable agentic coding over a significant footprint of our codebase. And all this is happening while the plane keeps flying, with near zero tolerance for defects in critical systems. There is a lot more to it than you will understand if you just look at twitter or Hacker News.

Not a single mention of value or revenue.

Only three metrics went up: LOC, token spend and presumably release speed/frequency.

What's the markup (profit/COGS)? is it up?

Customer satisfaction? Is NPS moving one way or another?

This post is funny to me, because I feel it falls well short of what a proper, forward-thinking "AI-native" engineering org would actually look like. That is, the author is presenting this as kind of a radical departure from normal engineering, but I just see it as normal engineering with more AI. I don't see that as AI-native.

To me, true "AI-native" engineering will be one where questions regarding how humans will do things like review code are irrelevant. You explicitly throw away any concept of typical software engineering and, instead, differ to the AI. Your job, as a human, is solely to do what the AI can't physically do. That's not the same as closing gaps between what AI is currently capable of doing and what you'd need it to be capable of doing. It's literally enforcing that anything an AI is able to do is treated as correct in itself, and you, as a human, only support it by being a human.

If that sounds outlandish, that's because it is. We don't have such capable AI systems yet. It's possible we never will. But that's beyond the point. The reality is trying to make an "AI-native" engineering org by just layering AI on top of existing processes and systems is futile. If your expectation is that AI will continue to improve in capabilities at an accelerating rate, then these stop-gaps are pointless. You might as well just ignore AI altogether until it is able to completely take over, because that's the inevitable goal, right?

If you're willing to buy that will be possible, then my guess is that it will occur in the form of small scale startups and actual mavericks rather than someone trying to run an existing business who will simply be too risk adverse to go all in when the time comes. Some guy in his basement is more likely to leverage AI "properly" to build an actual "AI-native" engineering org because that person won't have much to lose.

It this point, a whole lot of teams have been leaning into AI-first, not-entirely-reviewed code since around March this year.

That means we are already 6-8 months in, and not far from having a full year of experience of what that actually looks like, albeit weirdly influenced by the increase in model capabilities during that period.

I'm looking forward to those retrospectives. Did this turn out to work OK for the teams doing it? What were the biggest failings and/or surprises? With hindsight, did this produce good quality software?

I was just looking at some user-facing error messages Claude generated for me which contained unnecessary/irrelevant technical details while failed to provide clear instructions or fix.

Earlier, I reviewed code that does way more than specified scope, changes public API and introduces new behavior. I had to almost redo the whole thing.

I thought to myself, no way I am going to ship the code as written without actually reading through it (or as some people put it, only looking at the "intent"). I'll get more complaints from users while having absolutely no idea what's going on with the code or actual behavior.

It's a puzzle to me how these orgs can pull it off.

I see a lot of builders (myself included) falling into the trap of focusing on the wrong things. Development velocity doesn't matter per se, what matters are things like growth in profit or traction. It's really easy to conflate automation/systems/process with actual business results. Movement ≠ Progress.

Related: this article was very unilkely make it to the Hacker News homepage thanks to the clickbait headline, but I recommend reading it anyway: https://twitter.com/jsnnsa/status/2115623382721126826

It describes a team spending 30x payroll on tokens - $3,000,000 in token spend per month - building a multiplayer game where incidents and bug reports from users are passed directly to Fable 5/5.1 agents who then act on them and deploy the changes to production.

"We consider it an outage if we don't ship every hour"

It's worth reading because it's either a complete disaster or a glimpse of the future, or quite possibly both.

Spent few minutes reading the infomercial -- regretted it. Engineering principles form the bedrock first, and AI-tools are made to fit in them, not the other way around as this infomercial propounds. Its linkedin vibe is a big turnoff too.

Maybe it's just me, but "How much we ship per engineer" doesn't seem like the correct metric to track.

If that's the benchmark, developers are incentivized to produce over-engineered and bloated code.

Is there any way to read any of this if I don't have a twitter account?

For the argument lets say that this approach works and results in high quality apps. But will be users happy that app is updated 3 times a week and all updates add major changes? Because code generation is so cheap and fast it means we should even more thing about what to build to not overwhelm our users and actually solve their problems.

Nice work. How do you handle swapping in a new model version, hot reload or restart? And what happens to a batch that's mid-run when it changes?

Here we are again, thirty years after I was taught the "waterfall" paradigm at business school.

When asked about "customer satisfaction" we were told, awkwardly, that became "political"

The paradigm did not survive its encounter with reality, and I do not think this one will either

The "only" 175% increase in productivity is very impressive, and seems realistic to me, when you remove the chores of writing code (thank you to coding agents)

What you build is still very dependant on what you need, and you do not know that until you build it

It is a paradox only for the "waterfall" paradigm. We rejected it, in the 1990s for a reason

It did not work

In my best impression of the slapchop guy:

First you take your Blendtech and "will it blend" your engineers, then upload the result to the cloud inside what is called an S3 bucket (don't forget to set access permissions on your buckets kids!) to be digested by the clankers, sit back and profit!

Dark humor aside I think disconnecting a bunch of problem solvers from their computing equipment could really enhance the world, putting that analytical power into the world in person could move a lot of cogs, rather then marshalling JSON.