> Doomer argument is just that optimizers gonna optimize

Right. I don't quite subscribe to the strong doomer argument, but take water:

Water flows downhill. It's not conscious. I don't think it's anthropomorphism to say that water "seeks" the lowest point on the landscape. Often that seeking process is mostly harmless, sometimes it's devastating, and that's where you want levees and dams.

Now expand the same mathematical intuition out to other lowest-point-seekers in a high dimensional space. You don't even need to postulate a mind per se to see how optimizers can cause quite some mischief.

Meanwhile, Kids and Kittens are good examples of this behavior at the intelligent end of the spectrum: Given time, they'll find a way to get through any gap. They don't need to be as smart as the adults, the adults just need to turn their back for one minute too many. O:-)