Ask HN: How to deal with shipping AI-written code you don't understand?

I’m surprised I haven’t see anyone say this yet, but I don’t let the LLM design the code at all, I let it implement a design I have created.

By that I mean that, rather than let the LLM do the entire structure, I take the time to create the full hierarchy of abstractions and actors that I want my code to do. I decide, at the UML level, how abstractions will sit and interact — and then I make the LLM implement those gaps.

The key thing here is that, by doing this, I DO know how the code works. I may not know how each unit is implemented, but I know each unit’s responsibilities and how they interact with one another. I’ve found that LLMs are HORRIBLE at writing decent abstractions, regardless of how much prompting and skills I’ve given it. It will always get the job done, but it will often be unmaintainable and require bigger and heavier refactors as I introduce more and more requirement changes.

But by flexing my basic design skills, I don’t have those issues. Future refactoring fit into easy abstractions that I’ve designed. And by documenting it up front, I also have what becomes a living document on how this work will turn out.

It takes a few hours up front to plan ahead, but it makes the LLMs job easier and use less tokens, and it makes the code better long term. I don’t care if one unit is suboptimal, as long as the abstraction is right — because at worst I can change an implementation for a better one later.

Testing.

Lots and lots of testing. I usually interactively test with the LLM. Now that I'm spending less time, coding, I get to spend more time, testing.

In my experience, I test more, with an LLM, than I did, with my own code. I don't find coding errors, but I do find implementation errors.

On another note: I ran an LLM on a bunch of really mature code that I wrote, and could swear was "perfect," and it found a number of issues.

Nobody's perfect. Trust; but verify.

I’m slowly coming around to the idea that if it works, I’m good with that. I never used to personally check all my colleagues work, either, I just trusted that they’d done a good job.

That’s not to say I blindly trust an LLM - my project is an autovectorising compiler. I have it write a bazillion tests, and I have real, non-trivial programs that it compiled. I have it run a ‘gate’ before releases that takes literally hours on 5 different architectures. To say this code is tested is … masterful understatement.

I think what helps is a long planning session, going into details, working out corner cases, designing test-cases, you know work and effort. Once that’s done it’s easier to let it work through the night and do a major production feature.

I still review the code, but hand on heart I couldn’t say I’d reviewed everything, sometimes I miss something in a complicated check-in. I try to keep checkins small and contained, but sometimes that’s not easy to manage alongside the “don’t break stuff” rule as well.

So it comes down to: I’ve been working on this for 6 months or so, and so far this has worked out for me. You know, kind of like getting used to a colleague’s idiosyncrasies over a period of time.

I will say that in my experience the code quality is fairly high. Maybe that’s down to the effort put in ahead of time in the planning stages, I don’t know, but I don’t seem to suffer from the ‘slop’ that people complain of. YMMV.

Ask the AI how it works!

It's totally fair to load up some open source project into an agentic IDE and instead of asking it to write something ask questions like "how does X happen when I use feature Y?" or "can you explain how client code could accomplish Z"

All the time I see an AI use a language feature or library feature that I never used before or I see some code I don't understand and it does much better than a lot of humans do at explaining what it did.

This is how every product manager feels every day.

If you're an LLM prompter shipping products, you're outputs are more akin to a product managers, not an software engineer's.

Technically it's just a role change. You're now the project manager or something, not the developer. It isn't a project manager's job to understand the code, only to ensure it does what it should and doesn't do what it shouldn't. And to ensure the developers, etc str unblocked and doing well. Embrace the new role.

It's not too dissimilar to life as a Principal level engineer pre-AI. You technically oversee the work of many engineers on their own systems, but you couldn't possibly review all the code yourself. You appreciate that human code review reduces, but does not eliminate, the number and severity of bugs shipped to production. You don't have the low-level details in your head, just the high-level ones. You design how all these engineers interact with each other, without stepping on any toes.

Time will tell whether we were right to move from a system where we trusted humans to understand the low-level system, to one where we trust fancy calculators. But so far, it's working.

The traditional answer is the same for engineers going into engineering management because managing a team and managing a set of agents is highly similar, only managing agents is far easier. One only ever has the illusion of control, and so you have to set your mind to containing the worst possible outcomes and looking at the net efficiency and efficacy of the team with the understanding that they will, on average, never be perfect.

I lost my ability to fully explain the systems I work on around the time I transitioned from next-edit-prediction AI running in my IDE (early Copilot) and working primarily via Codex/Claude CLI's.

The further away I got from the implementation details, the fuzzier the semantics became. To the point that now I could not in good faith say I understand the system I have built from the ground-up and worked on the last ~4-5 years.

The solution to this, in my mind, is not to revert back to reviewing implementation details. The economics of that from a business/productivity perspective don't play out well.

The only viable solution in my mind is adapting code/languages themselves to the shift by making programs formally verifiable. That way you can be sure that the code "does what it says on the tin", and provably so.

I feel so strongly about this I've written on it:

https://gavinray97.github.io/blog/design-by-contract-and-eff...

1) Need to triple down on infrastructure (safeguards, tests, sandboxes, etc.).

2) Ask agents to do more than just code.

They need to fully demonstrate and prove that their PRs are rock solid. Ask for screenshots, videos, code reviews, detailed architecture diagrams, etc.

It would be an insane amount of work for humans, but AI can do it easily for each PR.

Merging any PR from an agent should be a no-brainer if there's real proof that the feature is built correctly and working.

If there's still doubt, then the agent needs to do a better job of proving that the feature is built correctly, doesn't introduce regressions, etc.

I'm miles away from all of this as a non-user of AI and it seems like a mania to me.

I was struggling with this today. Working on a feature and AI review flagged something so I need to fix that. I iterated multiple times throughout the day and was completely exhausted cause each iteration was always complicated with the fix being ~1k lines. Finally landed on a one I felt at least a little comfortable submitting for review but a work day like this is super draining and I felt like crap the entire evening (I am about to go to bed, I am probably gonna have nightmares about pressing enter and enabling auto mode on plans that make no sense in a true nightmare sense)

I can think of so many times this has happened before. People getting used to MVC, Dependency Injection, ORMs. Sometimes the code is so large you only work on a part of it, so you have to get used to a lot of not knowing.

So how do we build confidence anyhow when we can't know everything?

I think first, standard checks - does it compile? What do static analysis tools say - code quality, passes SAST analysis? Linters? Style checkers?

Do the tests run? I'm not comfortable with the tests I write let alone the AI.. by adding fault injection (fuzzing, chaos), and mutation testing, I have a better degree of confidence in the tests and code.

Spend even more time getting to know the product instead. Maybe I don't know the code as well as I used to, but I know the product backwards and forwards. That helps me have some new confidence that I didn't really have before. Knowing the greater whole.

Well, it's simple. Don't do it.

My Jira tickets are written by AI started just feeding the ticket into AI (shrugs)

No I only do that for unit tests so far

I went through like a mid-life crisis when the thing I learned/spent last 13 years on became meaningless.

This kinda makes me think about the transition from writing machine code by hand to using a compiler. Did people feel similar at the time that they didn't understand the machine code that the compiler was writing for them?

I've been spending a lot of time iterating on feature implementation plans with Claude (at work) and GPT (for personal projects).

I then make sure they write a lot of tests and that if functionality is changed or added/removed then tests are updated alongside.

Generally I try to review changes myself and ask it to explain the changes.

It's not perfect but I feel like the planning step being iterative isn't all too different from my old research -> think about implementation -> implement loop.

At that point, reviewing any code changes goes hand in hand with how I coded before LLMs.

The major difference is I usually have it make any changes for things I don't like.

the guidance / guardrails, agent roles, etc are the critical thing now, it's kind of like meta programming AI to get the outcomes you want. Any problems need addressing at this level. You don't need to understand all the code, but you should understand how the system is building up and adjust as needed. This would be an architectural review process you go through with AI. Features to code that should plug into existing architectural structures are generally no problem. It's just boilerplate code AI is generating. But architecture is only one aspect, you should have lots of guidance for all kinds of thing. There's security, performance, correctness, QA, Ux, Deployment, Process, Monitoring etc and having feedback systems for everything. You keep refining and reviewing your guidance and the impact it is having. As well as this you are molding the product for the things you want via feature development. What I'm not sure of is if you are some cog somewhere in a company like Meta whether whatever slice you become responsible for becomes a bit meh if you are mostly just prompting a feature and not much more. As a developer in a smaller company (we are global with a smallish tech team) I get to do a lot of things from interacting with our customers through to the entire backend and get to make critical decisions about nearly everything. Nearly everything in the company is starting to be driven via guidance and feedback loops (including on the guidance itself).

Test driven development and the testing pyramid.

If the agent has written tests and crucially - witnessed them fail before starting the work to make them pass - you have a lot of confidence.

Review of your own work's easier too - test and production code tend to mirror each other, giving you a much more complete picture. Confused by the production code? Go read its tests for a bit, then come back.

It's also easier to understand the change if you were there for the build, watching the transcript, checking in every so often. Doesn't have to be your primary task - leave it on a second monitor, it'll catch your eye naturally.

And finally, on top of test-driven development, have the agent first (before the first test driven development cycle) build out an end-to-end test covering the work it'll do. Once that test is green (test driven word for "is passing") you're done with your work and can clean up at will.

Play with it before shipping, get it real nice. It's never been easier to clean the tech debt up and make the UI perfect.

Eventually you'll be ready to package it up into a PR!

Now's you can leverage agents and get creative. Make your colleagues froth to review your work. Consider making a Loom video to walk them through it. Consider reworking it into a commit series; it only takes a few second for your agent and you can tell a story for humans. Maybe you want to sequence it according to your struggles; maybe it's better to show the different layers emerge with each commit - it's up to you. Consider also that your agent can easily do one-off visualizations of the change for your colleagues if you think it'll help them, and you can attach those too - mermaid charts or even whimsical AI generated images. All this is because motivation is driven entirely by other humans' motivation and you want them frothing with excitement every time you drop a review. Make it fun, it takes seconds and gets your stuff merged fast while making your teammates smile.

This is a brand new job and we will work out how to do it together :)

Flat-out, you _need_ to understand the code you're merging. If you don't, ownership then lands on whatever AI agent you're using. Ultimately, you can't fire the AI agent in the same way you can fire the person at the driver seat when shit hits the fan and they've consistently caused incidents that force people to work late hours to resolve it.

Maybe the new meta should be reviewing new code at the end of the week/month rather than at each merge? But even that might be too much of a bottleneck. The alternative is to talk to your agent to understand changes at a high level without reading every line of code.

if you have to maintain it, you will still have to get an understanding of it when it breaks.

not understanding the detail is quite similar to a coworker writing the code and another coworker approving and merging the changes.

It’s very disconcerting. I think this will largely be solved by going all in on AI and use it to test and review the code in an adversarial manner. That won’t be cheap but it’s the only way I see this work. But a lot of companies will probably save money and use AI mainly for writing code and then skimp on review and testing. I bet we will see some huge blow ups with AI written code and hopefully the industry will learn.

It also took the industry quite a while to accept that server code needs to be secure even if that requires a lot of extra effort. AI will probably be the same.

I've been playing with this https://getchronosphere.com. It will (attempt to) rebuild a PR in a coherent journey rather than a massive blob. It's been somewhat helpful up to this point, and still improving.

I love using AI, but any feature it creates I test the hell out of it, i become the QA testers I hated and do all sort of ways to break the app, and after 100 or so hours, I have a bug free and fully working app, now will this app scale as I add more features that to be seen.

> it all feels like a black box.

I have some work in Lean4 that I'm certainly unqualified to validate; Layer Coding QEC derivation, Positive Grassmanians and the Fresnel-Kerchhoff correction to the Parallel-Axis theorem, Huygens intensity as an alternate model for simulating photonic and electronic interference in a double slit experiment, for example.

It was a black box but I'm letting it tutor me about it so

Visual proofs, Tools for logic and formal methods in distributed systems

Things that compile logic!

> That means you can end up with systems you've built where you don't truly understand how all the code works

Though AI coding and system development can be overwhelming due to information overload and rapid growth in complexity (and concern for unwittingly embedded mistakes, the unsecurability of agent-based development, and number of LOC of code per day), not understanding the whole system sufficiently is normal and expected in engineering.

I think the solution is leaning on static types and hard to abuse abstractions to make programs correct by construction. It does mean that you’ll still have to use your brain to design the internal code architecture, and just use LLMs to fill in the boring bits between the safe abstractions.

Force to always use TTD, at least validate the tests.

Recommend reading (All though a little self serving): https://www.linkedin.com/pulse/ai-code-quality-jorge-varas-a...

Im currently building a ui framework for unreal. I dont enjoy ui work so I wanted a way to have to write less ui.

For me I dont look too closely at the implementation but I look at the api where if a human was using my framework how is the code they'd have to write and engage look at. Then I also do plenty of performance against a comparable target in my case slate (default unreal) and keep overhead minimal.

So good performance and reasonable public facing api are the 2 main pieces im focused on. Less with the internals.

Care about the seams and interaction and the things you can measure or count.

Formal verification, pick a language that has it integrated and prove relevant post conditions.

AI still does not know how to even write code that makes impossible state unpresentable so testing won’t help.

I'm currently working on a combined Aero sim/CAD/CAM software for designing, simulating, and performing DFM optimization specifically for Group 1 to 3 winged UAVs.

What I do is ask Opus/Gemini to explain the parts I don't understand. And I do some of my own research to verify accuracy of the explanations.

In addition:

* Every new feature has a Feature Design document which I review, modify, and approve. And a corresponding Implementation Plan which I also review and approve.

* Every fix has a RCA document and fix Implementation Doc which I review, modify, and approve.

* There's a global high level Architecture and Design document with additional sub component design documents which are always kept in synch after a new feature or bug fix is done. Synching it is always a step in the Implementation plan.

This does adds some overhead but it allows me to have have check point gates. This has often allowed me to catch misunderstandings and align properly before any code gets edited by the AI agent.

Having the above docs also helps when it comes to referencing them as context to properly ground new features or bug fixes.

I shipped code I didn't understand when I copy pasted from Stackoverflow before. At a level if it works it works.

>This new standard of shipping with AI is moving so fast that if we're going to spend our time checking each line of code, it's going to take forever.

"This new way of piling bricks is so fast that I can no longer afford to check the buildings integrity".

I just don't understand how the fuck we ended up with this narrative. Somehow the machine making things twice as fast forbids us from spending time ensuring quality? Surely if it does things faster you have more time and tools to check!

I'm half convinced this frenzy is the marketing work of AI companies at this point.

An engineer from Meta not knowing what they're doing is exactly how I imagined engineers at Meta. Thank you for confirming it's an actual reality.

On the other hand, not even surprised anymore most people even try to provide serious answers to this. Welcome to the pure insanity.

That's the neat part, you don't.

Or at least, I don't think people should.

To me this completely goes against the whole point of engineering.

IMO If you don't understand the code, then you're just a manager now. Soon your salary may also reflect as such, assuming you get to keep your job.

Product doesn’t care. And most of us will get replaced by agents. I guess around 30% of the engineering workforce will keep their jobs which will be mainly to keep the product up and running, doing whatever it takes (so using agents to fix stuff basically).

The ones who get to keep their jobs will be rebranded as Maintenance engineers with a lower salary. VPs and Heads of will keep their cozy job (bc there are not so many of them), and they are not that dumb to fire themselves.

Code understanding doesn’t matter anymore (or won’t matter soon)

I had same problem, but after a while I just accepted that is inevitable

By taking responsibility of it, maybe by saying "Hey this is not a good idea", and if that doesn't work perhaps by switching jobs if that's an option?

The mental gymnastics developers are willing to go through just so they can avoid standing up for something is quite perplexing.

Claude to Prod. Every day.

>How are you handling this?

easy, don't write code with LLMs. If my job forced me to do so I would quit.

It's simple: you don't. You slow down. It's essential to understand what you ship, before you ship, as there are real world consequences for not doing so, ranging from wasting the time and energy of yourself and others, up to failures which cause a great deal of harm to people and the world. To knowingly ship code you don't understand is grossly negligent, regardless of how common it becomes.

I don't ship code I don't understand. I fully review every line if it's going to prod.

My dumb little utils in ~/.local/bin? Those get merged after a quick functional test without my having read a single line of it. Still, the skill I maintain for such a job directs agents to write such throwaway scripts in anything but shellscript because I might want to extend some day and fuck shellscript

I deal with it by not doing it.

> How to deal with shipping AI-written code you don't understand?

Stop shipping code you don't understand.

Is there really any other solution? You can either ignore it and pretend like it's the new era of software like half the other commenters are, or you can sit down and understand the code and what you're shipping.

You just stop caring. Everyone else did about three months ago. Time to find something new to care about.

Generally speaking I brain storm the feature first and produce a dev plan. This includes asking it to reuse code/libraries we’ve already got! After it’s gone off ah done its thing we go back over and do a QA on what it’s built, why that approach, what can we do better etc etc?

I spend time to add extra observability, cross-checking and alerting at runtime to offset the risk.

It feels like another layer of "conceptual compression". I'm not sure who/when that phrase came around but in context of Rails is where I heard it and its things like ActiveRecord or really any ORM. You aren't writing the SQL directly, something else is and for the most part it works. Then it doesn't work or your database grows too much and you need to optimize table structures with indexes, etc.

My point is if you don't understand the fundamentals of the underlying code (SQL in the example above), you're more likely to have issues in the future. Or maybe not if that app doesn't reach a usage pattern or complexity where the nuances under the hood matter.

It really depends on what you're shipping. Meta ships social media apps where in the grand scheme of things if something is broken software wise it largely doesn't matter. What's the worst case? lost ad revenue? Users can't do things? If that were software for MRI machines or aerospace where software issues/bugs can result in lives lost then it matters much more in understanding what is going on at all levels of the software and potentially hardware.

You are accountable.

You're going to have to understand it one way or another, either at writing or at debugging. Even if you delegate all debugging to AI, you'll still need to determine what is "right".

If you don't understand the code at writing or debugging, you'll pay the architectural price (where it degrades). That price may be worth paying, but that depends on the system you are building.