alt Hacker NewsPrevent cognitive debt by manually retyping LLM-generated code
Comments
Lets prevent cognitive debt by manually retyping the assembly that is generated by our compilers...
"Prevent sub-optimal code by manually retyping compiler-generated assembly"
I don't think this is a practice which will be sustainable for very long
I already wrote my opinion on this, which I don't think anyone read, but my idea is to let AI code the working system, and then prompt it to teach you, give you challenges, and grade your work.
If you write, you should write in your own words, to demonstrate your own understanding - the so-called Feynman technique. Never verbatim. That's as true for coding as it is for study notes.
This is the workflow that Vs Code Copilot does. All the AI generated code changes are in a git worktree and you can step through them all. This is what I missed after Claude forced third parties to start charging API pricing and now I have to use Claude directly and I have to do this same review process in a clunkier way via my git client.
The most I enjoy working with AI is my special workflow.
I ask it to plan the feature in a separate worktree.
In parallel I start coding without being biased by AI and vice versa.
At some point I read its plan and iterate on it all the while I am in implementation mode. This helps me improve my own vision.
Finally I ask the AI to review my implementation. It flags off bugs and gaps which are usually straightforward for it to fix.
Whenever I don't know something, I ask it for a tutorial, programming-magazine style. Then I just follow the tutorial.
At that point I feel like you might as well do the implementation yourself and just plan with an LLM
At this point, just write the code yourself with your brain.
I wonder how effective it finally will be. At first glance it reminds me painting by numbers a d I'm not sure if that will help the real painter to keep his skills and surely won't teach aspiring painter much about the craft.
If you can afford it, why not. For certain phases of projects like a proof-of-concept, you need to move fast and validate several ideas. Once it's locked down, rewrite from scratch, and here, if you can afford it, type or write the code manually.
This is not slightly comical, it is very comical. Side projects are not mandatory, if you are using them to learn something, asking LLMs to generate and for you to type it makes no sense - just do it on your own. If you want to offload tedious boilerplate part to LLM, then by definition no need to learn it as it is tedious, so typing it out is useless.
the article's advice is to type the code yourself instead of pasting it. we used to call that programming.
The Zed Shaw method!
Nice workflow! I'll give it a try. I'm struggling with building mental model of AI-generated code. And code review fatigue is real. This may be the way.
I like the "cognitive debt" term. With the latest models, what I've observed is that they are really good, but I don't use them to write main code because I need to know what I'm doing.
The article is not wrong though that it pays off to have some imagination on how to use the models. For example, I want to use SIMD instructions in an ESP32-P4 CPU. Those instructions are undocumented for the most part, with just a couple of handwavey blog posts and some infuriatingly vague marketing material. So I just asked an LLM to create a `SIMD_P4.md` document with all the details. Lo and behold, it practically reverse-engineered the ISA. Now I can program in assembler by hand all I want and build that skill in my own brain, and whenever I find a slightly unclear op in the document, I ask the LLM to refine the documentation in that op.
It's better than nothing perhaps, but reminds me of UK highschool in the 80's (is it any different now?) where we had to manually copy everything down that the teacher was writing on the blackboard rather than the teacher giving handouts so you could pay attention to the teaching. The act of copying everything down was a negative rather than a positive.
Of course agentic coding tools are not trying to peer code or teach/inform you what they are doing, so being present in the moment doesn't help, but I suspect that copying it all down later doesn't help either.
When you are/were developing software without AI, even for pretty large projects you do end up internalizing (memorizing, but not deliberately so) a lot of detail, but from my own experience I'd say it's more the design than the code. The design is what you put effort into, thought about, etc, so is both what you naturally end up memorizing, and is what you need to know to have a mental map of the project and therefore understand how best to modify it. The code itself was naturally always the last thing you did, and followed automatically from the design and module/component interfaces - not something you typically think much about other than while in the flow of just "coding it up".
By retyping LLM-generated code, it seems you are mostly going to be gaining familiarity with the wrong thing - the code and not the design. Memorizing the code is not going to help much in grokking the design.
If you copy/paste code from a teaching book, you will probably not learn as well as if you type it.
Typing itself is irrelevant, it is the timing spent, even if only seconds, pondering at what each word or syntactic element is and why use it.
Being slower does not automatically make you learn better, focus on the learning is what makes the difference.
If you don't have the opportunity to learn, the time to actually think, then a faster tool is not helping.
TL;DR: what matters is why you are doing something, is it solely to get the task done or is it primary to learn, or both?
My rule is: I only let AI code for me, I don’t let it think for me.
Since writing is thinking, coding is thinking since coding is writing. That means any time I am not certain how I’m gonna implement some feature or bug fix, I have to code it myself because that’s the only way I can force myself to think through it. Only when I get to a point where I’m line “ok I know exactly what to do now- all that’s left to do is type it out” that’s when AI can be employed - essentially as a autocomplete.
This is only for projects where I will be held responsible for outcomes and must understand how it works. For hackathon / personal projects, I vibe away.
I also use AI to brainstorm at the outset of the task when I don’t know where to start at all.
Is this inefficient? My take: no. It’s maximally efficient. Over the long term it gives me an edge over any teammates who just vibe code everything because I actually, you know, understand how stuff works. I become the guy who can save the day at 3am when the team’s business critical app goes down. I become the guy that gets pulled into meetings so the suits can ask “is this possible?”. I see opportunities and problems before my teammates because I have a relationship with our code and system that they never took the time to develop or think about.
Would one retype assembly language for C generate code?
Having LLMs write out their design and reviewing it seems more efficient. Have LLMs, maybe with a different model, check that the implementation meets the design.
Filing this away
This is really good advice that I've found to be true, especially if you have autocomplete turned on. At the same time, I'm privileged enough to have a fast typing speed and anything less than 60 WPM would make this practice very annoying. Thank you for sharing this.
Funny that there's another trending post titled, "Don't be a meat proxy," just above this proposal that we literally meat-proxy all the code.
Whenever I encounter an especially preposterous proposal like this one, I like to imagine a USMC Drill Instructor wandering into the open plan office and having an interaction something like this:
USMC Drill Instructor: "What the actual fuck are you doing?!"
Smelly Recruit: "Sir, I'm hand typing the LLM output. Sir!"
USMC Drill Instructor: "Are you fucking with me recruit?! I said I wanted a SASS App, not a typing tutor! Drop and give me 20!"
Smelly Recruit: "SIR, YES SIR!"
Thats nuts. You are fighting a tool that is supposed to offload that. Is like lets not use the power tool, but do it by hand so your muscles won't atrophy, but you should instead use it to free up your muscles for other tasks like better requirements, architecture, tests, UX design.
But this way you move way slowly even on personal projects, like you will not even get the basic UI for the app done in a few days? Is that OK for you?
When I got my first corporate job, I was placed in a group of 20 trainees in a rigorous COBOL course. We were given assignments and a schedule to complete them.
Most of us read the specs, then raced into the coding phase, hands to IBM mechanical keyboards. One guy took a different approach. He took a legal pad and pencil, and wrote his whole program on paper before he ever approached a terminal. He’d do his own bug checking and syntax checking, instead of having the compiler do it ( compiles took longer in those days, and required JCL ). He avoided the entire compile/wait/read-with-dismay/quickly-try-again loop.
He was one of the top students, of course. And a lot less stressed, as I recall.
I ask the LLM to generate the code but mask the last token. Then I do softmax and give it an answer. Pretty soon my perceptrons are more connected than ever.
I don't understand the concern about "cognitive debt". I frequently have to maintain code I wrote, or someone else wrote, weeks/months/years ago and I have NFI what's going on. Now I say to the LLM "tell me what's going on" and it tells me. I can ask it some follow up questions, and build up my understanding. It's SO much faster than grepping through the source myself and I can do it for as multiple issues in parallel. The notion of reading every line of code is absurd to me, the notion of RETYPING it beggars belief. Surely this is satire.
nah thanks;
my workflow:
- ask not only for a solution to a problem but also for specific code (= tell the agent about your mental model of the codebase)
- ask for small stacked 'PRs/branches' and review/refactor heavily also using the agent (= refine your mental model of the codebase)
First, using LLMs to do typo/grammar correction was very impressive. Then, I realized I don't like the feeling of having to proof-read the text yet again, because I couldnt never really be sure it would only edit what I expected. Looking at a diff isn't really what I wanted either. So I started to prompt it to give me a list of errors/corrections. Two advantages here: I don't have to fear the LLM subtly changing my text n ways I missed. And, I actually do learn from the fixes, because I manually have to go into the source and fix them by hand. Takes more time, but is way more useful in the long run.
Then, I basically discovered the same pattern with agentic coding. Sure, a refactor or a bit of vibing is fastest if you have it done by an agent. But then, there is this fight against deteriorating code quality/plain bloat. I noticed it is 1. cheaper and 2. way more productive to just send the current translation unit source to the LLM and prompt it with a question. Then, go through the answer and manually copy useful code out. It is way easier to filter good from bad suggestions here, and it feels more productive iterationg. These days, I launch codex only to do changes that span across multiple files. But I feel it is way nicer to work on code without an agent, just using classic context. You control the context, the agent cant waste tokens by doing obviously useless research, and you end up reviewing the LLM output way more critically.
I find that trying to organize things into lots of sub folders helps to reason about my code a lot. I feel like we almost need something more powerful than a folder. Like lets say I'm making a game. I would have a ball.py, ball_manager.py, ball.png, ball.json, ball.md, ball.wav for the sound. All in one folder called ball. Or maybe better to have it organized in subfolders ball/code/ ball/sounds ball/data and so on. Then it would be like a reusable object, that could possibly reuse across future games. And then lets say have a net, and you would do the same for net code and assets. The problem is that in practice it's not that simple, since you still need to have the layer that talks between ball and net. In the game I'm making that layer has ballooned to 40,000 lines of code, working with mostly Codex. Codex seems to be able to reason about it, but for me, it's grown to a size I can't do it anymore. Even though that layer is just mostly instantiating and updating other classes. It's still so large now. So not sure how to avoid that bloat. It's too easy to ask for new features and then you get all this code that become near impossible to manage later for one person. Not sure how this is done on large coded bases in real world. But it seems like I needed to start with another abstraction from the get go that would initialize and update objects, so then ideally I would have like a list of objects and some abstraction so they could communicate with each other.
I feel like this will do almost nothing?
Mindlessly typing something is not much better than copy and pasting?
I could maybe see it if you asked it to spit out pseudocode you had to rewrite. At least there’s some translation there…
But this is bizarre. Write it yourself at that point. Is it any faster (or faster at all frankly) to prompt what you want, manually write it out, and maybe even make adjustments as you go? I’d argue not.
The way I wrote code in the past was to just first comment out what I wanted to do, and then underneath write the syntax for it. You could maybe do this too? Take the LLM code, and go through commenting what each section does to be able to effectively break it up? It still seems dumb.
better: before submitting the code, ask LLM to quiz you for understanding the code.
I don’t disagree with this if you code for a hobby.
Buy if you code for a job, good luck justifying this to management. “Yeah Claude already gave me the solution, I’ll take the rest of the week to type it out”
I don't think it's bad to manually retype code as a way of learning.
Isn't a working program itself the best textbook? It's just a difference in learning methods. Depending on Stack Overflow is also a dependency, and searching for code on GitHub is also a dependency. How much dependency you allow is purely a personal difference, and it varies depending on your own study habits and learning style. Whether your learning method is superior or not likely depends on how your brain works.
People tend to think that the more painful something is, the better it is.
I don't deny that there are talented people who can read the manual and build everything from scratch. But I think that analyzing and rebuilding a working template step by step is also valuable.
I agree with the view that LLMs may cause cognitive decline. But if you go down that path, Socrates already criticized writing for weakening human memory. And how did that turn out? Books became a universal medium for knowledge. Then the internet came along. When Stack Overflow appeared, there was opposition, but it also had explosive adoption. LLMs are just the next step in that sequence.
If there is cognitive decline, I think there's also compensation in other areas. Using LLMs clearly causes some cognitive decline. And I think there are areas that need to be reinforced to compensate.
But having a baseline to work from—modifying already-working code—is genuinely helpful. I don't see what's wrong with using that as a way to learn.
I just give them smaller tasks
This method doesn't seem bad.
Realistically, LLMs write code much better than most people. In my domain, there are areas where I still write better code than an LLM, especially when it comes to physical constraints it might not understand, but there are far more domains where the LLM writes much better code than I do. In that sense, writing code with an LLM and keeping track of it feels more helpful than I expected.
Practicing solo coding for an hour a day often ends up being mechanical and not very useful. This might actually be more helpful.
Cognitive debt is an idiotic phrase.
In my current workflow, I've settled into a three tier system when coding:
1. HIGH-VALUE CODE:
I write it all myself. I will occasionally use AI for mostly mechanical changes, like cleaning up variable names or mass-changes when a function signature has changed. Either way, every line is read carefully. Sometimes this means isolating my high-value code as a library in a separate repo. Usually it's just a note in AGENTS.md, or even a well-written comment at the top of certain files. I'm not obsessive about it, though, as it can't hide from git. And learning what it's trying to change is sometimes a useful insight.
That doesn't stop me from using AI as a consultant. This is the one time I'll use a beast like Fable. Ask it to write a technical/security analysis on a section of code and damn it can pull out some impressive insights. It can't write new code particularly well, but it can inspect code like a boss. But that all stays in the chat window. (And despite being so infrequent, they ends up costing significantly more than all my other AI costs combined!)
2. BOILERPLATE/PROCEDURAL CODE:
I'll write the first draft, but once I've set the tone, I'll allow AI to build and maintain it. I keep on top of things like a senior manager, just to make sure it's not doing stupid things. Every few days I tell it to mow its own grass: AI is good at recognising its own stupidity, you just need to give it an opportunity to look.
3. TEST/HARNESS CODE:
Bring on the slop. If I get nothing else from the AI revolution, it's not having to write another stupid test unit. Nothing makes me happier than setting the AI to work writing every permutation of test I can think of. I will slop this code all day, and I won't read a single line of it. Why should I? If I ever doubt whether a particular test is correct, I'll test the test by breaking the code, not by reading the test. But I almost never catch it out. In my experience, AI is especially good at writing tests. Perhaps more than anything else.
Tests don't just take the form of a few mocks and props in a test harness. In one recent case, my project involved writing a library for the API of an obscure commercial microcontroller-powered device. I took the API documentation and made AI build me a complete simulator. I then made it write a full suite of tests using my client library within the test code. I then got it to run that test suite against real hardware and identify any inconsistencies. From there it could recursively modify the simulator until it became unreasonably good at mimicking the real hardware. I haven't read a single line of its code. But it's now core to the library's CI.
Seems like a much better idea would be to build it yourself first, then have the AI do it for real? If you don't understand what you're building, you're going to get a lot of surprises when you re-type the AI-generated code and realize it subtly mistook your prompt or made decisions you didn't think to specify.
...what, that's terrible advice. If you gonna waste time on that just waste time on writing the code from scratch
[flagged]
[flagged]
[flagged]
[dead]
[flagged]
[flagged]
Jack Kerouac famously sat down and typed "Anna Karenina" on a typewriter because he wanted to feel what it was like to "write a great novel".