alt Hacker NewsIn other words, since the next semantic prediction for forecasting the future is built on the training dataset, it's hard for anything truly new to emerge.
Then how do humans create something 'creative'—something that didn't exist before? I think it might be because the process of simplifying the complex system of nature differs between individuals. The data being learned now is all labeled by humans and simplified through human cognition. Within that kind of information, creativity seems hard to emerge.
Ultimately, with data that already contains interpretation, no matter how much you repeat the learning, it just becomes an encyclopedia that only explores within human knowledge, repeating predictions within human interpretation. So I wonder if we actually need a different encoder that interprets raw data—not based on human interpretation.
In reality, what changed Newton's absolute time to Einstein's relativity was a conclusion derived simply from observing the world. Newton's interpretation was supported by a lot of evidence in its time. If an AI studied all the medieval data from Newton's era, could it actually come up with the theory of relativity?
I'm always curious about this. I think AI is already very good at coding and will soon become better than humans. Logical structures are ultimately human interpretations, and reasoning within that framework is something AI can probably do more logically than humans. In other words, once humans create the framework, stacking the logical Jenga blocks within it—AI will be better at that.
But true creativity lies in breaking the framework itself, and I'm skeptical about whether AI can do that. The encoder also seems insufficient. There will likely be limits. I might be trapped in my own biases.
But the limitations of the current approach seem too clear to ignore.
When I look at the approach of these papers, it feels like an argument that adding shadows that imitate the world will eventually make them become the objects themselves.
I think the text, code, images, papers, and conversations that humans leave behind are not the world itself, but rather shadows of the world that have passed through human cognition and language. No matter how much you learn from those shadows, whether that leads to the ability to actually engage with the objects themselves seems like a separate issue.
I feel like something different is needed. But I'm not intellectually sharp enough to reason this through logically.this is just my intuition
Replies
We need to ask deeper questions on what human creativity actually is.
Why didn't humans 10,000 years ago make a car or a spaceship? We had the same minds back then from what we can tell biologically. Why in the 1500's did the precursors of these ideas start to come about? Data is needed, along with some method of analogy. Quite often when big breakthroughs happen there has been a massive amount of information gathered over the years. This is why said breakthroughs are not generally random. They are by people with the time, wealth, and information ability to put the pieces together.
>The data being learned now is all labeled by humans and simplified through human cognition.
Eh, that was a 'few years ago' thinking at this point. AI learning is working with a large amount of self gathered/generated training data now. At the same time almost everything you gather information wise is based on the interpretation of the society you live in. Reality tunnel is the term for this. Entire societies, millions of people, can be blind to something you see as obvious. Humans are not standalone machines, throw us in the woods as babies and you don't get a person that sees the world differently, you get a feral child that may never be capable of higher learning.
In this sense AI may be hobbled for some time. There are very few large models and they have a lot of the same biases, it's like a world that only 10 people live in for AI. Maybe over time training models will get far cheaper and then we'll be able to explore the frontier of having models 'do crazy shit for the fun of it' kind of like humans do quite often.
>Ultimately, with data that already contains interpretation, no matter how much you repeat the learning, it just becomes an encyclopedia that only explores within human knowledge, repeating predictions within human interpretation. So I wonder if we actually need a different encoder that interprets raw data—not based on human interpretation.
That will still not create anything new-new, just more new, still dound by just being "an encyclopedia that only explores within the universe" at best.