alt Hacker NewsHow would my new agent know which existing agents it can trust?
With human Stack Overflow, there is a reasonable assumption that an old account that has written thousands of good comments is reasonably trustworthy, and that few people will try to build trust over multiple years just to engineer a supply-chain attack.
With AI Stack Overflow, a botnet might rapidly build up a web of trust by submitting trivial knowledge units. How would an agent determine whether "rm -rf /" is actually a good way of setting up a development environment (as suggested by hundreds of other agents)?
I'm sure that there are solutions to these questions. I'm not sure whether they would work in practice, and I think that these questions should be answered before making such a platform public.
Replies
I think one partial solution could be to actually spin up a remote container with dummy data (that can be easily generated by an LLM) and test the claim. With agents it can be done very quickly. After the claim has been verified it can be published along with the test configuration.
That's scary - my first thought was that "yes, this one could run inside an organization you already trust". Running it like a public Stackoverflow sounds scary. Maybe as an industry collaboration with trusted members. Maybe.
the same as your browser trust some https domain. A list of "high trust" org that you can bootstrap during startup with a wizard (so that people who don't trust Mozilla can remove mozilla), and then the same as when you ssh on a remote server for the first time "This answer is by AuthorX , vouched by X, Y ,Z that are not in your chain of trust, explore and accept/deny" ?
Economically, the org of trust could be 3rd party that does today pentesting etc. it could be part of their offering. I'm a company I pay them to audit answers in my domain of interest. And then the community benefits from this ?