alt Hacker NewsDeserved. word2vec is a beautiful peice of science. The hypothesis behind it was so amazing. I kinda feel like everything else in LLMs (including the attention mechanism itself) is just engineering around the representation learning breakthrough that word2vec demonstrated
And with applications far outside the original scope, which I think is one of the hallmarks of foundational discoveries. As soon as you have that one puzzle piece a whole lot of other stuff comes into focus.