"supercrawlers like this"

then in the post "It came from a single Amazon Web Services (AWS) address. It had read the main page of 18,926 projects, which is practically our entire catalogue, in 26,429 requests over a span of 34 hours."

sounds like some R&D dude running solo to be honest, nothing about that is a supercrawler lol