There is no reason to believe that generalist models need to be transformer (or transformer hybrids) it's just that people are risk averse and not spending their resources on the many scaled risky bets needed to explore the possibilities. There is more there.