> It won't unless there's another (r)evolution in the underlying technology / science
I think reinforcement learning with little to no human feedback, O-1 / R-1 style, might be that revolution.
I think gluing wings to a pig will make it fly. Show me examples or stop the conjecture.
There is lots of human feedback. This isn’t a game with an end state that it can easily play against itself. It needs problems with known solutions, or realistic simulations. This is why people wonder if our own universe is a simulation for training an asi.