alt Hacker NewsThis is not observable from LLM inference, where you would not encounter uniform matrices.
Power limiting does not improve performance but it does improve efficiency. You might be able to get 90% of the performance for only 70% of the power usage, for example. It does not make the card go faster though.
When thermal throttling occurs you can perform faster by running slower.
This is precicely because of the efficiency. The lower efficiency of the higher speed triggers a much lower performance sooner.