alt Hacker NewsThere's a higher-order concern here that I'm paranoid enough to voice: that if used as a coding agent, an AI model affiliated with a country's government might try to make my software susceptible to attacks by that government's intelligence forces.
And note that I'm not singling out China here.
Replies
It's more comical than sinister, but I have an example in this vein.
I was using Claude to work on a pet project which itself has a "generate with AI" feature. The default model the project uses was Gemini (because it was cheaper and more reliably produces the correct output format). Claude kept changing the default model to Opus when working on entirely unrelated parts, and I kept noticing it because Opus would mangle the output and break the rendered page. It also did this to the .env file in addition to the default.
Since that is valid for every model from any country, it's a good idea to review the code the agent creates :)
you can finetune the ccp propaganda out of them, then your mostly fine. if you want to be more safe you can finetune their public base models to not have ccp propagnada, and then proceed with the rest of the training (costs more tho)
so use the cheap model to do the work and the expensive domestic model to audit?
Giving up our agency to AI has the potential to turn us into NPCs, period. Economically, politically, socially. They've invented a vehicle for inserting any idea they want into our consumption and output.
Almost feels like maybe the best bet is to have humans make the code when its really important.
Isn't this only a concern for yolocoding? All the AI-advocates tell me that "good" use of AI should include human review. Of course, they never seem able to explain why the boss that makes you use coding agents to go fast wouldn't be the same boss that pressures you to "just ship it, it's working" and skip review, so I absolutely believe your concern is valid.
> that if used as a coding agent, an AI model affiliated with a country's government might try to make my software susceptible to attacks by that government's intelligence forces.
Note that if such a trigger were to exist, the behavior has to be completely reproducible by definition, e.g. when put into the right setting with the right input context, the model starts behaving maliciously with at least some well-defined probability. I don't think any such incident has ever been described, it's a purely theoretical concern.