I don't think this traffic is actually coming from crawlers for training.

Think about it, why would a training scraper need to hit the same page hundreds of times a day? They only need to download it once.

I think this is LLMs doing web searches at runtime in response to user queries. There's no caching at this level, so similar queries by many different users could lead the LLM to request the same page many times.