logoalt Hacker News

ModernMechtoday at 9:27 PM0 repliesview on HN

I'll try to explain my experience with this. I've noticed the AI has a tendency to overengineer scaffolding. For instance, I asked it to help me with a refactor, and it erected this massive 100kloc function registry, and then caused GitHub CI to verify the contracts every single commit, which took upwards of 30 minutes (I suspect this proclivity is widespread and has contributed to their recent issues).

As if this wasn't bad enough, it also was not smart enough to regenerate the evidence in these contracts as it changed the underlying source code. So it would get in a loop where it would update code -> commit -> 15 minutes later CI would error citing the contracts weren't updated -> it would fix the contracts -> 15 minutes later CI would error because the fix was wrong -> it would fix the fix and commit -> 15 minutes later contracts would fail -> contracts were fixed again and this time maybe 30 minutes later it would pass, maybe it errors again.

This loop could go on all day every day if someone wasn't paying attention because the agent has no concept of time or wasted work. It's an AI livelock of sorts, but it will eventually converge in my experience. It'll just take 10x longer (literally like 20+ hours) than if you just intervene and tell it knock it off, so it feels like lighting money on fire (hence the tax).

That's why I feel like this vibe coding stuff has to actually be monitored, like a Tesla system -- because like a Tesla system it cannot be trusted to not crash into the proverbial code wall.