alt Hacker NewsI feel as though you are confusing AI use in scraping by random companies and actual AI companies scraping. The AI companies seem to see value in walled garden sources like Reddit, Stack Overflow, etc. However, I don't think there has been any major instance of a major American AI company doing aggressive online website scraping and not respecting robot.txt.
Per https://thelibre.news/foss-infrastructure-is-under-attack-by..., all of the major American AI companies are not respecting robot.txt and participating in the AI-fueled DDoS of the internet.