What about AI not respecting robots.txt? I myself have never ran into this, but I've seen complaints of many people who did.
"What about AI not respecting robots.txt?"
since when actor that want gather your entire data respect things like this??? how can you enforce such things with just "please don't crawl this directory thanks"
alt Hacker News
"What about AI not respecting robots.txt?"
since when actor that want gather your entire data respect things like this??? how can you enforce such things with just "please don't crawl this directory thanks"