I very much agree and it saddens me often when I see folks dial it up to 1.5x or 2x "because it is boring otherwise". They might think they just get as much, only in half the time, but I'd hypothesize that they retain less because the mind needs some time to digest.
Similar with the trend to auto-cut pauses that speakers make, just to cram in more text into a youtube video. Destroys not just the pacing but any chance for the mind to take its time.
I'd love to see studies about this.
I don't think you can make blanket statements. Consider:
1. Teacher 1 speaks at 100 words per minute.
2. Teacher 2 speaks at 50 words per minute, for the exact same content as Teacher 1.
You're not really losing anything by playing Teacher 1 at normal speed vs Teacher 2 at 2x speed. For every learner, there will be an optimal teaching speed for a given topic. This may mean listening to some audio at normal speed, some at higher speeds, and some even at slower speeds. I always customise my playback speed to the specific content I'm trying to learn.