Agreed. That’s really my thoughts.
The internet or rather the crawlable WWW itself has changed.
The WWW content itself is no longer what it was.
Do you think the proliferation of web frameworks makes it harder to crawl?
Do you think the proliferation of web frameworks makes it harder to crawl?