logoalt Hacker News

zeitg3istyesterday at 5:37 PM1 replyview on HN

In the first image you can see how indexed pages go from 40k+ to 11 in the matter of days. Further down the thread I show how 114k+ pages are marked as “crawled but not index” and we can’t understand why. The rest is stuff that is (correctly) blocked by robots.txt.


Replies