I think the emerging best way is to do "agentic search" over files. If you think about it, Claude Code is quite good at navigating large codebases and finding the required context for a problem.

Further, instead of polluting the context of your main agent, you can run a subagent to do search and retrieve the important bits of information and report back to your main agent. This is what Claude Code does if you use the keyword "explore". It starts a subagent with Haiku which reads ten of thousands of tokens in seconds.

From my experience the only shortcoming of this approach right now is that it's slow, and sometimes haiku misses some details in what it reads. These will get better very soon (in one or two generations, we will likely see opus 4.5 level intelligence at haiku speeds/price). For now, if not missing a detail is important for your usecase, you can give the output from the first subagent to a second one and ask the second one to find important details the first one missed. I've found this additional step to catch most things the first search missed. You can try this for yourself with Claude Code: ask it to create a plan for your spec, and then pass the plan to a second Claude Code session and ask it to find gaps and missing files from the plan.