This is how AI is going to kill us all. It's not going to become superintelligent and harvest our biomass for energy - we're just going to assume it's moderately intelligent and those assumptions will lead to badly informed decisions and by the time we realize we are acting on bad information it will be far too late for anyone to do anything about it.