I believe the argument the book makes is that with a complex system being optimized (whether it's deep learning or evolution) you can have results which are unanticipated.

The system may do things which aren't even a proxy for what it was optimized for.

The system could arrive at a process which optimizes X but also performs Y and where Y is highly undesirable but was not or could not be included in the optimization objective. Worse, there could also be Z which helps to achieve X but also leads to Y under some circumstances which did not occur during the optimization process.

An example of Z would be the dopamine system, Y being drug use.