Unfortunately, even a holdout set doesn’t protect you from overfitting, it just takes longer.

Of course having a holdout set is better than not having one. It’s just not a silver bullet.