Agreed. A former mentor of mine told me a nice way of viewing software development:
1. Make it work. 2. Make it fast. 3. Make it pretty.
Transformers & LLMs have been developed to a point where they work quite well. I feel as though we're at a stage where most substantial progress is being made on the performance side.
The Huggingface transformers lib is currently undergoing a refactor to get rid of cruft and make it more extensible, hopefully with some perf gains.
A similar concept dates back to 30BC: https://en.wikipedia.org/wiki/De_architectura
Firmitas, utilitas, venustas - Strong, useful, and beautiful.
Heh, seems people I've been learning from been biased away from beauty, as I know that as "Make It Work, Make It Right, Make It Fast".