This isn’t quite accurate. Data weighting is quite important in pretraining.