Real search engines respect robots.txt so you could just tell them not to enter Markov Chain Hell.

jsheard • last Thursday at 2:48 PM • 1 reply • view on HN

throwaway744678 • last Thursday at 3:12 PM

I suspect AI crawler would also (quickly learn to) respect it also?

➕ show 2 replies

alt Hacker News