It's all OpenAI / Anthropic / Singapore/China based crawlers. I guess they gain data that I provide for free. I get very little crawling by Googlebot / Bing etc which barely even index my site, I only have a single page "indexed" by Google.

Possibly content that appears only briefly? I'm not sure otherwise. It seems wasteful.

Have you ever use an AI chatbot? It's basically a democratized scraper.

Type something stupid, it "visits" 200 websites and regurgitates some random gibberish.

The most recent info of every single fucking thing on the web. Everything. Data locusts.