In a sense, they already are. The giant leap in self driving cars we've seen in the last handful of few comes from using transformer models.