alt Hacker NewsTaking AI out of the equation, surely these same kinds of mistakes have been made in the recent past. As in, not doing due diligence and mistakenly killing civilians
(setting aside when civilians are purposefully harmed which is a conversation about morals rather than the capability to make correct decisions)
What I am getting at is, has AI really made the problem worse or has the mechanism just changed?
We are increasingly seeing "Sorry out <Black box> made a mistake, the developers are fixing it" rather than any one person taking responsibility for the decision. This is a growing problem that is made worse by AI. Over-inflated trust in AI output is also documented issue; "The perfect billion dollar targeting system with a 99.9% success-rate (on its verification dataset)" just made a "fluke"; "Nothing could have stopped this tragedy"
When it was just called a "database query" or "algorithmic estimation", it was more implied that the actual decision is left to be made and that no analysis had actually been done yet. But when AI preemptively says "I've analyzed every source of data and come to the conclusion that this ship is carrying nuclear bomb material"; it really sounds urgent doesn't it? It probably doubled down when asked about it.
https://edition.cnn.com/2026/09/18/politics/us-military-ai-f...