Its called legal or mandatory deposit. In some countries their national library is required to crawl the open internet within their language, besides collecting regular published materials. In the US laws on legal deposit has not been extended to non printed materials.