logoalt Hacker News

Decoupled DiLoCo: Resilient, Distributed AI Training at Scale

45 pointsby metadatyesterday at 4:57 PM5 commentsview on HN

Comments

SilverElfinyesterday at 5:32 PM

Is this actually innovative? I respect that there’s a lot of work in making it reality and doing it specifically for AI training by modifying their algorithms. But doing portions of work in clusters that are far apart and combining them has been done many times before for non AI things, right? Or so I would think.

show 2 replies
SubiculumCodeyesterday at 9:34 PM

This is potentially scary, national security wise