alt Hacker NewsIt's worth pointing out that this is a multi-trillion (probably) parameter model optimizing an 800 million parameter model. There is no indication that it can improve itself with this same methodology.
It's worth pointing out that this is a multi-trillion (probably) parameter model optimizing an 800 million parameter model. There is no indication that it can improve itself with this same methodology.
If that multi-trillion model is composed of thousands of MOE experts, each expert model might be small enough for it to comprehend.