alt Hacker NewsI think models could be designed that in separate layers created "logical system" representations which could feed back into the output, much like how attention works. Attention is about relevance, the logical layers could be based on logical schema-based patterns.
Consider an LLM that happened to have some pre-trained layers that were trained abstractly on all the constructive proofs available for modern mathematics. LLMs with image recognition rely on existing visual pattern recognition layers, fwiw.
There's another blog post that made it to the front-page of this site which sums up the state of the art nicely [0].
It's not obvious that they will be able to do any reasoning, in the formal sense, at all; let alone better than humans. LLMs are simply not sufficient for the kinds of tasks and work done when reasoning about mathematical problems.
There's plenty of research demonstrating that they can be useful in small, constrained tasks -- which isn't anything to raise our noses at!
... it's just not _obvious_ in the sense that there is a clear step from LLM capabilities today to "better than humans." It's more an article of faith that it could be true, some day, if we just figure out X, Y, Z... which folks have been doing for decades to no avail. In other words, it's not obvious at all.
[0] https://garymarcus.substack.com/p/llms-dont-do-formal-reason...