alt Hacker NewsBecause the transformer architecture that enabled modern LLMs wasn't invented until 2017[1]?
1: That's the "T" in GPT fyi, even though Google is the author of the research paper that changed everything
Because the transformer architecture that enabled modern LLMs wasn't invented until 2017[1]?
1: That's the "T" in GPT fyi, even though Google is the author of the research paper that changed everything
Right. So we had enough information to train LLMs but not the technology to build it.
So the initial models arent just distilled from information. We’ve always had the information.