The problem is that they wouldn't be able to talk to any internet service that legitimately used those addresses. Whether that was likely to be a problem very much depends on whose addresses they were.