Yes. There as difference between unfair and unreal; someone who is malnourished when growing up will forever likely be weaker than someone who received a proper sequence of meals.

We should perhaps recognize that and try to compensate for it, and it's not a value judgement on the person so afflicted, but pretending it doesn't exist just confuses matters.