alt Hacker NewsI've been working on an ML model capable of robust continuous learning, resistant to catastrophic forgetting without relying on replay, an external memory system, or unbounded parameter growth. Last week I confirmed the first non-toy, 580M parameter version soundly beat LoRA, EWC, and full fine tuning. This week I'm scaling up to 4.4B parameters...
Do you have a public repo for that? I'm also trying to do that although I'm using "replay"/distillation and hopfield memory banks.