Because the crawlers dont care, they are the internets parasites. Their creators care nothing for people or systems downstream of their greed.