alt Hacker NewsIs Apple silicon really that impressive compared to LLMs? Take a step back. CPUs have been getting faster and more efficient for decades.
Google invented the transformer architecture, the backbone of modern LLMs.
Replies
xmcqdpt2 • today at 1:35 AM
You can understand how transformers work from just reading the Attention is All You Need paper, which is 15 pages of pretty accessible DL. That's not the part that is impressive about LLMs.
> Google invented...
"Google" did? Or humans who worked there and one who didn't?
https://www.wired.com/story/eight-google-employees-invented-...
In any case, see the section on Jakob Uszkoreit, for example, or Noam Shazeer. And then…
> In the higher echelons of Google, however, the work was seen as just another interesting AI project. I asked several of the transformers folks whether their bosses ever summoned them for updates on the project. Not so much. But “we understood that this was potentially quite a big deal,” says Uszkoreit.
Worth noting the value of “bosses” who leave people alone to try nutty things in a place where research has patronage. Places like universities, Xerox, or Apple and Google deserve credit for providing the petri dish.