I think firmly marrying llms with symbolic math calculator/database, so they can check things they don't really know "by heart" would go a long way towards making them seem smart.

I really hope Wolfram is working on LLM that is trying to learn what it means to be WolframAlpha user.