logoalt Hacker News

fragmedetoday at 7:24 PM0 repliesview on HN

https://firehose.directory/robots.txt isn't one, but comes up in Google despite that.

https://atproto.at/robots.txt is good and findable with a direct search

https://atproto.wiki/robots.txt is empty but findable on Google.

https://pulsar.feeds.blue/robots.txt is 404 and thus not indexed by Google.

https://leaflet.pub/robots.txt is good. A logged in google search for this finds this for me.

Tell the people at https://pulsar.feeds.blue to fix their robots.txt if they want to get indexed.