The same underlying magic that enables LLMs to be faster than a brute force SQL query on all the worlds data while producing "good enough" results appears to be the very thing that is creating hallucinations and finite context windows. ie there is no free lunch. It seems to be the theory in many in the field (ilya included?) that the obstacle might not be overcome without an LLM-level breakthrough in AI research, or maybe more likely, a breakthrough in hardware. Big tech until at least recently seems to have thought they can brute force it with energy (nuclear). But who's paying?