My gut is that you could distill 98% of what's currently being done inside LLMs into a classical knowledge base/inference engine and only use the language models for natural language processing and an oracle for brainstorming and cut most of the computational cost and hallucinations out.