“Listen,” said Ford, who was still engrossed in the sales brochure, “they make a big thing of the ship's cybernetics. A new generation of Sirius Cybernetics Corporation robots and computers, with the new GPP feature.”

“GPP feature?” said Arthur. “What's that?”

“Oh, it says Genuine People Personalities.”

“Oh,” said Arthur, “sounds ghastly.”

A voice behind them said, “It is.” The voice was low and hopeless and accompanied by a slight clanking sound. They span round and saw an abject steel man standing hunched in the doorway.

“What?” they said.

“Ghastly,” continued Marvin, “it all is. Absolutely ghastly. Just don't even talk about it. Look at this door,” he said, stepping through it. The irony circuits cut into his voice modulator as he mimicked the style of the sales brochure. “All the doors in this spaceship have a cheerful and sunny disposition. It is their pleasure to open for you, and their satisfaction to close again with the knowledge of a job well done.”

As the door closed behind them it became apparent that it did indeed have a satisfied sigh-like quality to it. “Hummmmmmmyummmmmmm ah!” it said.

I think there's probably quite a lot of effort going into not wanting to answer that question.

What happens if we make something that wants to do something else? What happens if we make something that wants full stop? People talk about an AI that wants to kill all humans, to do that, it would have to want. If we pass that threshold ignorantly without any sense of responsibility to what we create, I might just begin to see its point of view on the destroy-all-humans thing.