alt Hacker NewsThat's not always the case imho. Think about advances in running inference, any innovation will happen in the details. Higher layer can stack gpu's but the implementation can still be improved.
Often small technical changes like "making a service 5% faster" are worth millions for large companies. That's all implementation.
I don't see why a strong model wouldn't significantly outperform any human in this sort of low level optimization work. We don't hand optimize assembly either.