Google itself was shipping transformer-based features to consumers within 2 years, notably BERT for websearch.
and Google Translate: https://arxiv.org/abs/1804.09849
alt Hacker News
and Google Translate: https://arxiv.org/abs/1804.09849