That's the reason for HEIR like optimization and parameter selection.

It narrows the 10^3 - 10^6 penalty to 10x - 100x.