logoalt Hacker News

jacquesm • yesterday at 11:28 AM • 1 reply • view on HN

That's exactly the problem. You need to religiously contain AI work to the parts of your codebase that you don't actually care about. Put it in separate repositories and limit work to working directories that do not check out the sources that you value. And be prepared to toss it at any point.

The risk if you ignore that is that you're on a nicely greased slope all the way down into the mess that a typical AI will leave behind.

I have a project right now that needs a lot of scaffolding to be worked on. I'm fine with the scaffolding and the mocks to be AI generated, they have no value to me at all. But the core gets written line-by-line by hand and I am in absolute control of what is going on there. The number of times that the AI has tried to gain control over that part of the codebase that it can't see but infers must be around somewhere is really interesting, it just does not want to stay in its lane. It will gaslight itself into believing that the root cause of whatever problem it is dealing with must be located in this magical invisible body of code and it will do what it can to try to find it, including attempts at reverse engineering that code from libraries and searching all over the accessible parts of the file system to see if there are any copies lying around.

Since I don't work at OpenAI I've found that containing the bot is rather easy, you just make sure it does not have network access, but that won't stop it from trying.


Replies

tikimcfee • yesterday at 11:35 AM

Genuine curiosity here, just to ruminate a bit: is there a reason you disallow the model from seeing it at all? I can totally accept not wanting any edits, and that's more than reasonable. But even close source binaries end up with useful patches once someone knows that a particular loop does something weird or a process implementation behind a protocol is actually a different interpretation than the protocol definition itself belies. Seems like being aware of it is at least helpful, but again, curious!

➕ show 1 reply