logoalt Hacker News

chaara-devlast Thursday at 4:04 PM3 repliesview on HN

You can already do this with a robots.txt file


Replies

mqustoday at 12:01 AM

> If other pages point to your page with descriptive text, Google could still index the URL without visiting the page. If you want to block your page from search results, use another method such as password protection or noindex.

> If your web page is blocked with a robots.txt file, its URL can still appear in search results, but the search result will not have a description.

https://developers.google.com/search/docs/crawling-indexing/...

So, a robots.txt will not keep your site off of google, it just prevents it from getting crawled. (But, to be fair, this tool probably does not do this as well)

btbuildemlast Thursday at 4:17 PM

Technically speaking, yes - but it's in no way enforced, as far as I understand it's more of an honour system.

This malicious solution aligns with incentives (or, disincentives) of the parasitic actors, and might be practically more effective.