alt Hacker NewsNorbert Wiener in 1960:
"As is now generally admitted, over a limited range of operation, machines act far more rapidly than human beings and are far more precise in performing the details of their operations. This being the case, even when machines do not in any way transcend man's intelligence, they very well may, and often do, transcend man in the performance of tasks. An intelligent understanding of their mode of performance may be delayed until long after the task which they have been set has been completed. This means that though machines are theoretically subject to human criticism, such criticism may be ineffective until long after it is relevant. To be effective in warding off disastrous consequences, our understanding of our man-made machines should in general develop _pari passu_ with the performance of the machine. By the very slowness of our human actions, our effective control of our machines may be nullified. By the time we are able to react to information conveyed by our senses and stop the car we are driving, it may already have run head on into a wall."
"In neurophysiological language, ataxia can be quite as much of a deprivation as paralysis. A patient with locomotor ataxia may not suffer from any defect of his muscles or motor nerves, but if his muscles and tendons and organs do not tell him exactly what position he is in, and whether the tensions to which his organs are subjected will or will not lead to his falling, he will be unable to stand up. Similarly, when a machine constructed by us is capable of operating on its incoming data at a pace which we cannot keep, we may not know, until too late, when to turn it off."
Source: https://www.cs.umd.edu/users/gasarch/BLOGPAPERS/moral.pdf
Replies
What a paper!
And you missed an even MORE relevant excerpt!!
Man and Slave
The problem, and it is a moral prob-
lem, with which we are here faced is
very close to one of the great problems
of slavery. Let us grant that slavery
is bad because it is cruel. It is, how-
ever, self-contradictory, and for a
reason which is quite different. We
wish a slave to be intelligent, to be able
to assist us in the carrying out of our
tasks. However, we also wish him to
be subservient. Complete subservience
and complete intelligence do not go
together. How often in ancient times
the clever Greek philosopher slave of
a less intelligent Roman slaveholder
must have dominated the actions of his
master rather than obeyed his wishes!
Similarly, if the machines become
more and more efficient and operate
at a higher and higher psychological
level, the catastrophe foreseen by
Butler of the dominance of the ma-
chine comes nearer and nearer.
Maybe they didn't have proper debuggers in 1960? For a language model you need (RNG state, context, prompt).
So if they wrote an LLM step by step debugger, it would be all deterministic. But they prefer rapid sales, chaos and mystique.