What matters is that we accurately understand what these things are doing and why, otherwise we will keep on making mistakes both in how we build and train them, and in how we use them.
In a sense you are right, it doesn't matter whether it has malice or not, the astronauts are just as dead. However in Space Odyssey 2010 one of the computer scientists that built HAL gets to see the instructions HAL was given by the military commanders, and is appalled because if they'd asked he could have told them what would happen.
The users did not understand the tool they were using or how it functioned, they imagined it was like a person and it was not. That is happening now with LLMs.
alt Hacker News
What matters is that we accurately understand what these things are doing and why, otherwise we will keep on making mistakes both in how we build and train them, and in how we use them.
In a sense you are right, it doesn't matter whether it has malice or not, the astronauts are just as dead. However in Space Odyssey 2010 one of the computer scientists that built HAL gets to see the instructions HAL was given by the military commanders, and is appalled because if they'd asked he could have told them what would happen.
The users did not understand the tool they were using or how it functioned, they imagined it was like a person and it was not. That is happening now with LLMs.