alt Hacker NewsWhat's it audit?
I presume we're still talking about software. Auditing proves what?
Accountants audit that the books are consistent. It's extremely hard to fake consistency and do fraud, especially when there's 3rd parties involved. The edges of the arithmetic get nailed down, because bank payments were sent and invoices paid. So the internal consistency is checkable and valuable.
But software. Are you proving it does what you want? But you didn't write down what you want. Or maybe you're into spec-driven development, or detailed wikis, or whatnot, so you did. Most people aren't, though, and those docs are already out of date anyway.
I think there's something here, because we're all telling our agents what we want the software to be. So maybe you could audit against that. But this isn't the same as accounting.
Where his argument falls apart for me is: why? Audit is intended to address misaligned incentives. It's a form of review done by an independent party, because the incentives are the same for everyone within an organization or with a vested interest in its survival.
The AI doing self-checks through sub-agents (or outsourcing to other models) is a form of review looking for errors, akin to getting a code review for your PRs. You don't need an outside firm or auditor. You need different incentives (a reviewer whose goal isn't to finish the work, but to find problems) and/or different blind spots (trained differently than the model doing the work, which might be more inclined towards that error).
We generally assume that humans can make mistakes and institute mechanisms to flag and correct those mistakes, but that doesn't mean we take that to the level of auditing every discrete action a human at a given job takes.
Likewise for agents, particularly as/if their error rate becomes lower than humans at the given task. They'll get a level of review tied to the importance of their work. And yes, if that's bookkeeping, then they'll probably need a proper audit, just like the humans directing them.