See the recent development of DiLoCo at Nous Research and Prime Intellect.
oh that's very cool, exactly the kind of stuff we need
alt Hacker News
oh that's very cool, exactly the kind of stuff we need