logoalt Hacker News

gavinray • today at 5:12 PM • 1 reply • view on HN

Didn't Google's "sleep" paper already show this?

https://research.google/blog/the-sleep-framework-a-new-way-f...

  > During the offline sleep phase, the model unlocks pre-allocated Mixture-of-Experts (MoE) memory blocks and updates its numerical weights using self-generated practice examples.

Replies

The-Ergster • today at 5:29 PM

I think this is much more significant that that. The article suggests more advanced weight adjustment

➕ show 1 reply