alt Hacker NewsAn AI Vibe Coding Horror Story
Comments
Meanwhile on Linkedin… Every sales bozo with zero technical understanding is screaming top of their virtual lungs that evrything must be done with AI and it is solution to every layoff, economic problem, everything.
It is just a matter of time when something really really bad happens.
Every other field that's figured out high stakes failure models eventually landed on the same solution - make sure two people that understand the details are looking at it - pilots have copilots surgeons with checklists and nuclear plants have independent verification. Software was always the exception, cause when it broke it mostly just broke for you, vibe coding is not going to change the equation, it barely removes one check that existed before is that the people who wrote the code understood what was going on, but now that's gone too
I think vibe-coding is cool, but it runs into limits pretty fast (at least right now).
It kinda falls apart once you get past a few thousand lines of code... and real systems aren't just big, they're actually messy...shit loads of components, services, edge cases, things breaking in weird ways. Getting all of that to work together reliably is a different game altogether.
And you still need solid software engineering fundamentals. Without understanding architecture, debugging, tradeoffs, and failure modes, it's hard to guide or even evaluate what's being generated.
Vibe-coding feels great for prototypes, hobby projects, or just messing around, or even some internal tools in a handful of cases. But for actual production systems, you still need real engineering behind it.
As of now, I'm 100% hesitant to pay for, or put my data on systems that are vibe-coded without the knowledge of what's been built and how it's been built.
> All "access control" logic lived in the JavaScript on the client side, meaning the data was literally one command away from anyone who looked
This is the top!
This is a typical example of someone using Coding Agents without being a developer: AI that isn't used knowingly can be a huge risk if you don't know what you're doing.
AI used for professional purposes (not experiments) should NOT be used haphazardly.
And this also opens up a serious liability issue: the developer has the perception of being exempt from responsibility and this also leads to enormous risks for the business.
Software engineering is looking more and more like it needs a professional body in each country, and accreditation and standards. Ie it needs to grow up and become like every other strand of engineering.
Gone should be the days of “I taught myself so now I can [design software in a professional setting / design a bridge in a professional setting].” I’m not advocating gatekeeping - if you want to build a small bridge at the end of your garden for personal use, go for it. If you want to build a bridge in your local town over a river, you’re gonna need professional accreditation. Same should be true for software engineering now.
I saw something very similar a few months ago. It was a web app vibe coded by a surgeon. It worked, but they did not have an index .html file in the root web directory and they would routinely zip up all of the source code which contained all the database connection strings, API credentials, AWS credentials, etc.) and place the backup in the root web directory. They would also dump the database to that folder (for backup). So web browsers that went to https://example.com/ could see and download all the backups.
The quick fix was a simple, empty index.html file (or setting the -Indexes option in the apache config). The surgeon had no idea what this meant or why it was important. And the AI bots didn't either.
The odd part of this to me was that the AI had made good choices (strong password hashes, reasonable DB schema, etc.) and the app itself worked well. Honestly, it was impressive. But at the same time, they made some very basic deployment/security mistakes that were trivial. They just needed a bit of guidance from an experienced devops security guy to make it Internet worthy, but no one bothered to do that.
Edit: I do not recommend backing up web apps on the web server itself. That's another basic mistake. But they (or the AI) decided to do that and no one with experience was consulted.
I know, through personal acquaintance, of at least one boutique accounting firm that is currently vibe-building their own CRM with Lovable. They have no technical staff. I can't begin to comprehend the disasters that are in store.
This reads like internet fiction to me. Very vague and short.
What would a responsible on-boarding flow for all of these tools look like?
> Welcome to VibeToolX.
> By pressing Confirm you accept all responsibility for user data stewardship as regulated in every country where your users reside.
Would that be scary enough to nudge some risk analysis on the user's part? I am sure that would drop adoption by a lot, so I don't see it happening voluntarily.
I interviewed some years ago for an AI related startup. After looking at the live product, first thing I see is their prod dB credentials and openAI api key publicly send in some requests... Bad actors will be having a lot of fun these days
I think it is wonderful.
It's reminiscent of the 90s, where every middle manager had dragged and dropped some boxes on some forms, and could get a salesman to sell it, without a care in the world for what was going on behind the scenes.
Until something crashed and recovery was needed, of course.
The piper always needs to be paid.
I believe there are various dimensions to vibe coding. If you work with an existing codebase, it is a tool to increase productivity. If you have domain specific knowledge, in this case - patient management system, you can build better systems.
Otherwise, you endup simulating the production. Lot of the non technical folks building products with AI Vibe coding are basically building Product Simulations. It looks like a product, functions like a product but behind the scene, you can poke holes.
Is there anybody making some framework where you declare the security intentions as code (for each CRUD action) and which agents can correctly do and unit test? I have seen a Lovable competitor's system prompt have 24 lines of "please consider security when generating select statements, please consider security when generating update statements..." since it expects to dump queries here and there.
Archived version:
https://web.archive.org/web/20260331184500/https://www.tobru...
Damn!!! And I keep hardening my RSS app which was partly vibe coded and not exposed to the WAN while "professionals" give data away.
I think the issue here is less about AI misbehaving and more about people doing things they should not be doing without thinking too hard about the consequences.
There are going to be a lot of accidents like this because it's just really easy to do. And some people are inevitably going to do silly things.
But it's not that different from people doing stupid things with Visual Basic back in the day. Or responding to friendly worded emails with the subject "I love you". Putting CDs/USB drives in work PCs with viruses, worms, etc.
That's what people do when you give the useful tools with sharp edges.
Who should get jailed ?
Does the company which willingly sells the polymorphic virus editor bear any responsibility, or should the unaware vibe coder be incumbent ?
I really hope OP also contacted their relevant national privacy authority, this is a giant violation
So much is missing from this story. Did they report it to the relevant data authority? Did the fix they said they applied actually fix anything? Etc.
[flagged]
There’s another version of the Mythos narrative that reads like:
AI companies realized that all this vibe coding has released a shitstorm of security vulnerabilities into the wild and so unless they release a much better model to fix that mess they’ll be found out and nobody will touch AI coding with a 100ft pole for the next 15 years. This article points more towards this narrative.
this is exactly the kind of vibe coding horror stories I asked for just few days ago :)
Anyone else read the title on HN and shudder not wanting to actually click it?
Don't blame the AI for what is clearly gross human negligence. It's like renovating your entire house and then acting surprised when the pipes burst because you used duct tape as a permanent fix.
Report them - that right there is 5+ different violations. Only then will they realize their stupidity.
A perfect example of why a product like Medplum exists, as opposed to completely reinventing the wheel from scratch
The only thing what helps is deleting the database. Every day. Until the thing goes down because the 'developer' thinks he has a bug that he can't find.
It's nothing new, dunning kruger existing long before AI entered coding realm.
Several years ago ran into one american company which consulted with me. They had 4000 paying customers and they rolled out their billing solution which accept crypto, paypal and stripe.
They had problem with payment going missing, i migrated them to WHMCs with hardening and they never had any issues after.
Now people may laugh at whmcs but use the right tool for job
U need battle tested billing solution then whmcs does count it can support VAT, taxes, reporting/accounting and pretty all which you'll error while you try to do it all yourself.
Too bad there aren't battle tested opensource solution for this
Kinda crazy but hopefully the future holds a Clippy-esque thing for people who don’t know to set up CI, checkpoints, reviews, environments, etc. that just takes care of all that.
It sorta should do this anyway given that the user intent probably wasn’t to dump everyone’s data into Firebase or whatever.
I personally would like this as well since it gets tiring specifying all the guardrails and double-checking myself. Using this stuff feels too much like developing a skill I shouldn’t need while not focusing on real user problems.
Hard to believe... This activity should certainly land you in a German prison?!
[dead]
Some people only care about actual consequences. Download all the data and send it, in the post on a flash drive, to the GDPR regulator's office and another copy to the medical licensing board because why not.
Avoid javascript like plague, it can be overwritten at the client side.
Do you think if the agency hired a consultant to build this , a consultant couldn’t have made the same mistakes?
Lack of security theater is a good thing for most businesses
I did something similar to a local company here in Spain. Not medical, but a small insurance company. Believe it or not, yes, they vibecoded their CRM.
I sent them an email and they threatened to sue me. I was a bit in shock from such dumb response, but I guess some people only learn the hard way, so I filed a report to the AEPD (Data protection agency in Spain) for starters, known to be brutal.
I've also sent them a burofax demanding the removal of my data on their systems just last friday.