alt Hacker NewsI wonder how the Chinese labs are training a 3 trillion parameter model on what has to be vastly smaller compute resources. If the U.S. compute advantage is persistent, it's hard to imagine that Chinese labs will be able to keep pace forever, as a matter of physics, but... so far they seem to be doing just fine.
It's not like same parameter count models are identical, so that doesn't appear to be an indicator for quality, or even compute requirements?
There seems to be more to producing a better model than brute forcing parameter count after all.