Yeah the assistance you get from a bike being light is huge. I wonder if expensive light bike > assisted bike in terms of saving human effort per trip.
Obviously if the e-bike requires no pedalling at all it wins but then that is not legal everywhere.
I think the main difference due to weight boils down to you riding in one gear lower when not using assistance.
On flat roads, I usually ride on 3rd or 4th gear (out of 7), now i ride in 2nd or 3rd.
But where i ride, the road isn't even and has a lot of steep slopes. There I'm on 1st gear all the time anyway and the assistance i get totally saves my knees, and ensures I'm not all sweaty when i reach my destination. Even compared to the light weight regular bicycles i rode before, this is better.
Human effort just isn't worth very much. The strongest humans on earth can manage about 400W for an hour. Even very small ebike motors are usually capable of 500w continuous/forever, so long as the motor does not spend too much time stalled or at very low speeds.
For a normal human, a "legal" 250w motor is easily doubling or tripling their normal power output, so hauling around an extra 20kg on top of the existing ~100kg body + bike is not a big deal.