logoalt Hacker News

looobay11/04/20251 replyview on HN

There was research on LLMs training and distillation that if two models have a similar architecture (probably the case for Xai) the "master" model will distill knowledge to the model even if its not in the distillation data. So they probably need to train a new model from scratch.

(sorry i don't remember the name but there was an example with a model liking howl to showcase this)


Replies

-_-11/04/2025

Subliminal learning: https://alignment.anthropic.com/2025/subliminal-learning/

show 1 reply