From what I understand, in terms of genetic changes to intellectual abilities, there's not much evidence to suggest we're so much smarter that your proposed teleported baby would be noticeably stupider - at best they'd be on the tail of the bell curve, well within a normal distribution. Maybe if we teleported ten thousand babies, their bell curve would be slightly behind ours. Take a look at "wild children" for the very few examples we can find of modern humans developed without culture. Seems like above everything, our culture, society, and thus education is what makes us smart. And our incredibly high calorie food, of course.
it's impossible to prove the counterfactual (I guess, as I imagine we don't have enough gene information that far back). But I'd imagine that the high calorie food you can get starting with the advent of agriculture is exactly what could drive evolution in a certain direction that helps brains grow. We've had ~1000 generations since then, that should be enough for some change to happen. Our brains use up 20% of the body's energy. Do we know that this was already the case during the stone age?
That is exactly what civilization is about - for new generations to start not from scratch, but from some baseline their parents achieved (accumulated knowledge and culture). This allows new generations to push forward instead of retreading the same path.