The last level is simply unused at this time.

https://go.dev/wiki/MinimumRequirements#:~:text=The%20Go%20t...

Diminishing returns apply to ISA extensions only on average, not for individual applications.

Most of the recent additions in processor instruction sets are intended for relatively niche applications.

In such cases, other applications will not be affected at all, but the specific application that is the target, for example a certain cryptographic algorithm or AI inference, may be accelerated many times when using the new ISA version instead of the old ISA version.

Moreover, it is frequent that compilers are not smart enough to take advantage of such ISA extensions, so it is not enough to change the compilation flags, but you need to rewrite some library to get the full performance benefit. For example, many recent x86_64 CPUs have IFMA instructions (integer fused multiply-add instructions), which allow the use of the floating-point multipliers for doing arithmetic operations with big integer numbers (the advantage is that modern CPUs have many more FP multipliers than integer multipliers). This can accelerate a lot the computations with big numbers, but you need a complete carefully-written library that uses such instructions, you cannot just recompile some programs for making them run faster.

From time to time it may still happen that some ISA extension has a wider applicability, being able to accelerate many applications, possibly just by recompilation, like Intel hopes to happen with the APX extension that will arrive early next year, in the Intel Nova Lake and Diamond Rapids CPUs.

Most non-professional computer users are biased toward single-threaded application performance, where diminishing returns have already been seen for more than 2 decades.

On the other hand for multi-threaded application throughput, we have not reached yet any diminishing returns. The throughput per CPU socket has continued to increase in geometric progression every year until now. The only serious problem is that starting around 10 years ago, from the days of Intel Kaby Lake and Coffee Lake, the price of computers has started to increase and the increase rate has accelerated recently.

So now the possible throughput for a given computer size becomes less and less relevant in comparison with the throughput per dollar, and for the throughput per dollar it appears that we have already entered the region of diminishing returns (i.e. with unlimited budget you can still buy computers whose throughput increases in geometric progression each year, but the computers that you can actually still afford have a throughput that increases much slowlier).