Yeah I bumped on that too. If it's possible to make llms cheaper than current grep, then it is also almost certainly possible to make grep cheaper.

LLM is spicy memoizing, so it can potentially be faster than a tool call. But people will spend a month tweaking and testing to ensure they have the level of determinism they need, which means it's more expensive, and that they should have used actual memoization in the first place.

What if the tool is more advanced, like an optimizing compiler: `g++ -O3 -march=native -x c++ - <<EOF ... EOF`

If the compiler invocation is sufficiently slow, the llm could consider outputting a binary directly?

For all we know matrix multiplications are a faster way to generate optimized machine code than branchy sequential compiler code with tons of heuristics and passes.

But grep is just a tool in a pipeline between a question in your brain and an answer you are searching for.

what if an LLM finds the answer early?

grep might continue to read everything, doing the wrong thing correctly.

> The author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep.

At some future point where LLM hardware is cheaper than simply running grep, then grep equivalent would benefit from those selfsame hardware improvements and be cheaper to run as well, probably still by the same ratio.

4-5 orders of magnitude is huge. Assuming an order of base 10, it's 10000x-100000x. So a call to grep may return in 1s on a typical PC. That means a GPT call takes equivalent energy of 10000-100000 PCs to do the same in 1s. That's a difference that can't be equalized with scaling. It would require a revolutionary breakthrough.

I also don't understand where the idea that frontier models are getting better efficiency comes from. The results are certainly improving, but that comes from feedback and multiplexing requests, which cost more.

A classic case of someone projecting out to infinity from just after the first bend of the S curve.

Right. And some hardware improvements will speed up both grep and Luna, which won't close the gap.

Well, think that statement through a bit:

Grep reads through the entire file looking for patterns.

An LLM scans its neural net (in ways that I don't understand) which is kinda-sorta like having a huge index.

You can improve over Grep if you have an index; and the LLM has an index.

Thus, it's plausible that an LLM can be more efficient at reading its neural net (IE, index) than Grep reading the whole file.

It might never beat out grep, but it could beat some more expensive to call tools, similar to how heuristics will often be faster than exact answers. Rust Analyzer can be slow at times, I could see an AI tool taking over a subset of its work.

> Barring a true breakthrough in something like quantum computing or room-temperature superconductors

Won't that also help grep and then move the asymptote down more?

> calling an LLM will soon be cheaper than a grep

From a computational standpoint this is obviously nonsense, but from an attentional one I'm not so sure. It may already be more attentionally expensive to use grep in some cases, such the moment you need to remember a non standard arg. And if this applies for performing a simple http operations, then it certainly applies going up the complexity chain.

room-temperature superconductors, sure, but I fail to see how quantum computing will disrupt – in the medium term (25 years or so) – classical computing in any meaningful way

Is running LLMs (or some other ML workload) on/with quantum computers expected to bring efficiency gains?

"Did you know that disco record sales were up 400% for the year ending 1976, if these trends continue...AY!"