alt Hacker NewsThe distinction I would make here is that computer code is logical transformations on arbitrary data, not the actual data itself. An LLM can learn the entire space of logical transformation patterns from existing code, and can hallucinate new logical transformations, using a computer as a validator for the logic, so an LLM can create new logic as well as repeat existing patterns, and that logic can be applied to novel input data that the LLM has never seen before.
That’s not how LLMs work at the moment as far as I understand. LLM would not hallucinate any new logical transformation, rather just predict a transformation from its training data.
I understand that there can be many different combinations for all the logical transformations in the training data. But still the number of combinations are finite and I would assume that large number of those combinations would not result in any meaningful outcome.
Best outcome is that it just predicts a new pattern we haven’t discovered (LLM randomly connected the correct dots) one example is protein folding.