alt Hacker NewsVaswani, A., et al. (2017) Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, 4-9 December 2017, 6000-6010.
Generally understood to be an output of Googlers.
Vaswani, A., et al. (2017) Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, 4-9 December 2017, 6000-6010.
Generally understood to be an output of Googlers.
Transformers are an applied science: https://patents.google.com/patent/US10740433B2/en
Basic research would be something like optimal control theory, which came well before the transformer design.
I'm not trying to be evasive; I can see how my distinction could be seen as conveniently just outside industry's purview. Put it this way: I think companies, particularly small ones, are incentivized to pursue well-known methods/materials. Innovation modulates and optimizes.