Not necessarily if you count capital costs vs operating costs/margins.
Replacing cars every 3 years vs a couple % in efficiency is not an obvious trade off. Especially if you can do it in 5 years instead of 3.
You can sell the old, less efficient GPUs to folks who will be running them with markedly lower duty cycles (so, less emphasis on direct operational costs), e.g. for on-prem inference or even just typical workstation/consumer use. It ends up being a win-win trade.
You highlight the exact dilemma.
Company A has taxis that are 5 percent less efficient and for the reasons you stated doesn't want to upgrade.
Company B just bought new taxis, and they are undercutting company A by 5 percent while paying their drivers the same.
Company A is no longer competitive.