alt Hacker NewsI see this line of thought put out there many times, and I've been thinking: why do people do anything at all? What's the point? If no one at all is even reviewing the output of coding agents, genuinely, what are we doing as a society?
I fail to see how we transition society into a positive future without supplying means of verifying systemic integrity. There is a reason that Upton Sinclair became famous: wayward incentives behind closed doors generally cause subpar standards, which cause subpar results. If the FDA didn't exist, or they didn't "review the output", society would be materially worse off. If the whole pitch for AI ends with "and no one will even need to check anything" I find that highly convenient for the AI industry.
Replies
I mean, the promise of perfect AI and perfect robotics is that humans would no longer have to do anything. They could live a life of leisure. Unfortunately, we're going to get these perfect AI and perfect robotics before we transition socially into a post-scarcity, post-ownership society. So what will happen is that ownership of the AI and robots will be consolidated into the hands of the few, the vast rest of us will have nothing economically relevant to do, and we'll probably just subsist or die.
We're already seeing this today. Every year, thousands of people are becoming essentially irrelevant to the economy. They don't own much, they don't invest much, they don't spend much money, they don't make much money, and they are invisible to economics.
You could e.g. write specs and only review high level types plus have deterministic validation that no type escapes/"unsafe" hatches were used, or instruct another agent to create adversarial blackbox attempts to break functionality of the primary artifact (which is really just to say "perform QA").
As a simple use-case, I've found LLMs to be much better than me at macro programming, and I don't really need to care about what it does because ultimately the constraint is just that it bends the syntax I have into the syntax I want, and things compile. The details are basically irrelevant.