I can’t attest to the entire past, but my ancestors on both sides were farmers or construction workers. They were fit. Heck, my dad has a beer gut at 65 but still has arm muscles that’ll put me to shame — someone lifting weights once a week. I’ve had to do construction for a summer and everyone there was in good shape.
They don’t go to the gym, they don’t have the energy; the job shapes you. More or less the same for the farmers in the family.
Perhaps this was less so in the industrial era because of poor nutrition (source: Bill Bryson, hopefully well researched). Hunter gatherer cultures that we still study today have tremendous fitness (Daniel Lieberman).
My dad was a machinist, apprenticed in Germany after WW2. Always somewhat overweight (5'9", 225 lbs during his "peak" years), but he could lift guys up by their belt with one arm, and pick up and move 200+ lb metal billets when he got too impatient to wheel the crane over. Even at 85 now, he's probably stronger in his arms than most 60 year olds. But I'm also not saying ALL of his co-workers were that strong, either.