alt Hacker NewsHow would you use tool-calling to filter through millions of documents? You need some search functionality, whether old-school search or embedding search. If you have only thousands of documents, then sure, you don't need search, as you can feed them all to the LLM.
I haven’t built either system but it seems clear that tool calling will be ‘O(num_targets * O(search tool))’, while RAG will be ‘O(embed_query * num_targets)’.
RAG looks linear (constant per lookup) while tools look polynomial. And tools will possibly fill up the limited LLM context too.