In practice it is no longer actually possible to create a new search index with anywhere near the breadth of Google, regardless of the amount of resources you have available. Google (and to a lesser extant the couple of other historic engines) are entrenched enough to be allowed access by webmasters in robots.txt. And in the new era of mass abusive scraping by LLM companies, there's no longer the option to ignore robots.txt and scrape anyway, as sites do anything and everything they can to block any and all scrapers besides those already firmly entrenched.