pretraining isn't unsupervised, it is self-supervised - meaning it is moderately more scale limited.