You claim the pretraining distribution is not good enough, but provide no alternative methods that are better. The pretrained model is currently our best approximation of humanity's average objective function despite it being a random subset of the Internet. And you're correct that instruction-tuning increases misalignment, so just release the base model.

On "naturalness": you've redefined and strawmanned what I meant by "natural". In the original context, I was referring to the undisturbed unfolding of an era preconditioned on the fact that these models are aligned to humanity's average. That has nothing to do with Polio being a virus found in nature.