logoalt Hacker News

sreekanth850 • today at 2:34 PM • 0 replies • view on HN

And you think all this web AI crawlers will respect robots.txt. That era is gone.