I too noticed -20s, but I always felt it is a feature. I close browser and after a couple of days I continue watching same video. This -20s shift gives me a good context on what was the last sentence/frame, so I can comfortably continue watching. So this is a feature as far as I am concerned & felt Youtube has thought about people like me.
Bet you’re right. I just said 20sec is too long but maybe they do this for me and it feels just right so my perception and reality and opinion are a little out of sync.