Google may be aggressively crawling for AI and only making a small subset visible to the search database.