logoalt Hacker News

nearbuyyesterday at 8:22 AM0 repliesview on HN

The landmark paper, "Attention is all you need", that triggered the breakthrough that led to current transformer architecture LLMs, only came out in 2017. Without that breakthrough, they wouldn't exist. And even then, the early models produced gibberish. Better gibberish than older Markov chain text generators, but asking GPT-2 "What is three plus five?" would give some nonsense, non-sequitur answer, that might start with a (incorrect) number if you were lucky. At the time, everyone was wondering if scaling up the model size would improve intelligence or hit a wall. ChatGPT didn't release until 2022.

And you'd need to know back in 2015 that Nvidia specifically would be the big winner from AI. They don't even manufacture their own chips. Intel also designs chips and GPUs, but if you bet on them in 2015, you'd have lost money between then and 2025.