depends on what you are grepping ... greapping a large file might be more expensive one day than generating n-th token with LLM that works fully in hardware

you could make hardware implementation of grep and store the file itself next to it in some ROM but that's not a very useful grep ... while hardware LLM is exactly as useful as software LLM only orders of magnitude faster