"You're absolutely right, that wasn't a military target—it was actually a girls school. It won't happen again!"

I realize that's not a great argument and was definitely tongue-in-cheek, but given there's still a lot of debate about the accuracy of AI for far more mundane tasks, my personal perspective is that until we have LLMs and such that are truly, demonstrably far more accurate than humans, with true reasoning and judgement capabilities, they don't belong where lives are at stake.

I wouldn't want an LLM-underpinned machine running anesthesia during a surgery; why would I want an LLM-underpinned military apparatus that is deciding the lives of far more? I wouldn't, not in their current state.

In a hypothetical future where we truly trust incredibly smart AIs or LLMs or whatever "smart" technology it is for driving weaponry, okay - if it's truly necessary; I abhor war and the death and destruction wrought by it.

In my mind, though - even if we get to that future where there's some vastly superior technology to the LLMs we have today, which can judge and reason, then I'll have a bunch of other questions, like understanding the motivations of said technology, because I suspect it'll be something much closer to AGI, and that opens a whole separate can of philosophical worms.

It’s true that humans are unreliable, even if we could guarantee a consistent set of moral and ethical standards as well as the appropriate tools and authority to enforce them at all times. We do, however, have to deal with 100% of the implications, one way or another. Machines don’t. Vasily Arkhipov and Stanislav Petrov had to consider the world they would condemn their children to if they pushed the button, the missiles didn’t. These are imperfect devices made by imperfect beings, whose rules and goals can be changed more or less at will by whomever wields them. Offloading the responsibility of pulling a trigger or trampling someone’s legal or human rights to such a machine while insulating those in command from consequences threatens to make things worse at a speed and scale I don’t think we’ve seen before in all of our dark history. Meanwhile, it further concentrates immense and unchecked power in the hands of a relative scant handful of elites, all of whom exhibit the lack of ethical and moral fiber we were talking about to begin with.

Teaching something or someone to pull a trigger is trivial compared to teaching them not to. I have no reason to believe that automated weapons and surveillance will be more reliable than our world leaders or that the world will be safer with them in it.

Mainly I don't want them because I don't want a machine instantly making life or death judgements. It might be fine if it mostly informs a human who ultimately pulls the trigger, but having that failsafe is important in keeping machines from going on killing sprees.

I also want people to be held accountable when they do unjustified killings. AI weapons make it FAR too easy to simply pass off a killing as a "woopsie doodle." It's just not acceptable to say "The algorithm made a mistake, version 23 will do better".

I don't have a problem with the AI providing additional information to it's user, but when that's incorporated into a weapon it's a short distance from that to completely automating the killing.

That's why I'm completely against AI weapons.

>Just out of curiosity though why do people not want smart AI weapons?

When the decision to kill another human being is made that should be in the hands of a directly accountable other human being, not an unaccountable machine developed in the basement of a private corporation.

And mines, both dumb and smart, in particular anti-personell mines are banned by the Ottawa treaty ratified by 162 countries. It's exactly the autonomous and fundamentally uncontrollable nature of mines, not just that they're dumb, that has produced countless of casualties long after wars were over. Can you tell me that millions of autonomous loitering munitions are not going to end up exactly like those mines still blowing legs off people decades after conflicts are over? And who is responsible then?

They aren’t training smart AI weapons to discriminate between civilians and soldiers. Even so, what happens when the soldiers are disguised as civilians? Or when the enemy forces civilians to serve or be a decoy?

Because it's going to be used in any possible way, and certainly not for the good of the people, as imply in your "naive" false dichotomy.