When the AI scrapers were just getting started, that is basically what I thought - their plan was to scrape / suck up everything they possibly could before people realized what was happening and blocked them.

The rate at which they were spidering and scraping was so far beyond what any other supposedly legit spider was doing, it seemed like the logical explanation.