There is evidence that coal has worse environmental impact than other fossil fuels. For one, burning gas produces CO2 and water, whereas burning coal results in just CO2 (+ soot and other pollutants). Another is that (open) coal mines have devastating effects on large land areas.

So yes, best leave all fossil fuels where they are, but coal is especially bad.