My gut says the economics, e.g hardware/data center/resource constraints, are going make the economics of small specialized models more attractive. Without any evidence whatsoever, I also think that the big frontier companies will have to de-emphasize chatbots as huge models in favor of chatbots as huge products with a much much more granular mixture of experts approach, but with much smaller models. I’ve been saying for a while now that AI products have to hit the gas on prioritizing product design to reliably solve real people’s problems in predictable-enough ways, because the current approach is only really appealing to enthusiasts, developers, or optimistic managers, and with the kind of money they’re throwing around, that’s not going to work.