Oh my god. It's the crawlpocalypse.

Most crawlers have no concept of what that is. They will follow links to this site and then follow links out of this site even after being told not to [1]. The majority of crawlers follow zero rules, RFC's, etc... The few platforms that do follow standards and rules are akin to a law abiding citizen in Mos Eisley.

[1] - rel="nofollow"

Unfortunately, the firebase API is very bad as they even acknowledge that in their github page.