The churn is an issue. For a while there, it felt like we were getting a new number format every month or two (e.g., fp4, ternary, etc.). That level of innovation works against moving things into hardware, or at least you need to be willing to spin hardware constantly.
Even then... Do we now stop the with to asic DeepSeek and do k3 instead?
Assuming such work was happening.