The Vibe Tax

i'm not sure why people expect agents to one shot everything to perfection with just a prompt.

There's a reason why we talk about software development lifecycle, design, architecture, testing ... It's because it's been the most reliable way to build and ship software. We shouldn't expect discard this and expect agents to perform well outside of this.

I'm treating LLM agents as junior devs who happen to have vast knowledge of software engineering. As their team leader i make them go through planning, implementation, bug sweeping cycles using strict workflows. And it works quite well, i've been working on several large projects (1M+ LOC java,typescript,c/c++) and by any measure the projects are healthy. Sure the code isn't that beautiful, sure i'd have written things differently but it's pretty good nonetheless.

Shameless plug here: i've been also working on https://kodfactory.com, the code factory i've built to work on these large projects with workflows, reviews, etc ... I'm cleaning things up to open source it later.

You can't one shot a perfect app with AI. You definitely can create a pretty complex and beautiful production-ready app with AI in a couple of days or weeks depending on what exactly you're building.

I now have my own link catalog, read-latter app and an RSS reader. Tailored to work exactly how I like. Hardened, with automated backup, and external users for the RSS app. It works. It takes learning, some knowledge of terms and very high-level practices, plus design thinking, but I haven't written a line of code for these.

I feel this, yes.

In effect, I’ve always wanted a pair programmer agent, not a zero to one programming agent. Unfortunately models these days are mostly of the latter kind and it has caused a major disruption in the way I work. I’d much rather appreciate a small model making fast and specific edits that I ask if it, rather than ingesting 20 files to make changes, and then starting to write tests, etc.

I feel like I'm living in a parallel universe when I read these types of posts.

My agents have never created code that is straight-up garbage and I have never flushed a week's worth of tokens down the toilet. I just can't identify with all the constant complaints about AI-assisted coding.

And my biggest project isn't some hello world app. It's a self-hosted, privacy-focused personal financial management application that I intend to open source. It's about 126k LOC against 240k LOC of regression tests and 30k LOC of CI/CD pipeline. I'm doing 24x7 mutation testing on a dedicated box against the accounting engine and temporal systems. I even have specialized agents doing audits against Regulation Z (US banking law) criteria so the app models the required behavior of banks.

Most of my complaints about everything are nits, like the overly verbose and dense way LLMs communicate with me. Or their predisposition to add, add, and add more stuff when proper engineering practices are more often about subtraction (but I've built mitigation guardrails against a lot of that).

I tried, but I’m not sure I understand. The vibe tax is caused by the model trying to one-shot everything and doing so requires unnecessary tests? How are vibe coders training the model over months? Do you mean their sessions and preferences are being fed back into the RL?

This article somewhat reflects my experience with autonomous agentic coding. I've run several experiments with similar results: the agent burns through all my tokens while making very little progress, or produces something unacceptable.

So I'd rather micromanage the process step by step. It takes more of my time, but the result is much, much closer to what I actually wanted.

I've found that LLMs make throwaway software better than I ever did.

They handle edge cases, catch bugs, and write tests that I'd never write.

Even if, however, this leads to the average piece of software improving, this one-shot complexity has the same issues as any large project. The more code, the longer it takes to steer the ship.

This "rising tide lifts all boats" mentality will make exceptional software even rarer than it is today.

Excited for the Roller Coaster Tycoons of tomorrow[0].

[0]https://en.wikipedia.org/wiki/RollerCoaster_Tycoon_(video_ga...

I’ve never had an agent fail to write the actual implementation. Has it done so badly, yes, but not nothing but tests. This sounds to me like a rare case that doesn’t generalize.

If the general idea is that these agents write too many tests, sure I guess? ‘Too many tests’ doesn’t sound like a failure case of engineering to me; typically software has had too few tests. Also, a lot of the power of these agents is their ability to self-verify and correct, which the test loop is a part of.

Nobody is making you pay this supposed tax. Just tell it not to write tests.

You can absolutely prompt agents not to write tests, or not to write extraneous asserts, or whatever, and I find that generally quite useful for the kind of code I write. I don't think it's "months of users training it", it's more that a lot of people do want a one-shot agent, and having a good test set really helps that.

This really resonated for me. It's like the smarter the model gets, somehow the more tokens get burned? Same failure mode whether you’re on Claude, Codex, or Cursor: the harness will spend the whole pool if you let it. I'm building my own Harness on top of pi that is add supported (www.freepi.ai) mostly because pi is so much more efficient with tokens. (That said, it tens to be slower and vastly more verbose with information I don't need to know). But yeah, since I'm trying to offer free ad supported inference the vibe tax would kill the business model. I've even been thinking about installing the 'caveman' skill to reign in token costs.

whenever I read these type of articles or comments where people are getting such bad negative experiences, I do wonder, what are they doing wrong or is there something that they are not sharing?

I've been able to get such positive returns out of LLMs. I am working for 3 different remote jobs concurrently with it, I've shipped a few apps thats doing six digits a month, I found a life partner after I used LLM to really work on myself. I am also experimenting with hardware prototypes and will likely have funding to launch it all with LLMs.

Why am I able to get so much out of "vibe coding" but others seemingly do not? I am not a genius, I am not a artisan, I am just very persistent and clear on what I ask LLMs but more importantly I don't try to place any other sort of unrealistic expectations on what it can and can't do.

You read comments on HN and read these articles and you might come across feeling a sense of peril and doom which are all completely fictional for the most part. A lot can be achieved with LLMs, much more than what the constant doomers will try to drag you down to.

I feel like these are two competing goals:

* the dev wants to describe an app in natural language then fall asleep while an AI works on it

* the dev wishes that the same AI would write less comprehensive tests

What exactly is a vibe coder to this dev?

Needs more info, has a good storyline but I am left trying to understand the overall pattern and trend implied there.

This is a confusing description of a real thing. They're clearly biasing the models more and more towards long horizon end-to-end software development, which leads to impressive "claude, build an X make no mistakes" demos, but is mainly an annoyance for expert users doing real work.

(If you give claude an inch these days it'll just steamroll through a whole program of work without checking what it should be doing - a kind of overenthusiastic pull towards the first draft that is often detrimental and definitely wastes tokens, and even for very basic tasks it's using many more tokens than it should because it's doing this full belt and braces thing for everything, just in case you're an idiot).

But also... it's something you can easily reign in if you want to.

Create a spec and have a dumb model execute it. Problem solved.

Pre-October 2025, maybe yes. But now? Couldn't disagree more. There is no insight in this post.

[dead]