The same underlying magic that enables LLMs to be faster than a brute force SQL query on all the worlds data while producing "good enough" results appears to be the very thing that is creating hallucinations and finite context windows. ie there is no free lunch. It seems to be the theory in many in the field (ilya included?) that the obstacle might not be overcome without an LLM-level breakthrough in AI research, or maybe more likely, a breakthrough in hardware. Big tech until at least recently seems to have thought they can brute force it with energy (nuclear). But who's paying?

No need to stress out over us rank and file answering that question. An entire economy is boiling based on it.