> If your stance is that luck would not ideally affect one's punishment
I don't think this is generally the stance. If you give someone a little shove, they bump their head, and they get a bruise, you've committed assault, you're facing up to a few months in jail. If you give someone a little shove, they bump their head, and they die, you've committed murder, you're facing potentially years in jail. According to the eggshell skull legal doctrine[0], it doesn't matter that some people are especially more vulnerable than others (ie that you were particularly unlucky and pushed someone who happened to have an eggshell skull), you take responsibility for the consequences when you do something illegal.
Now in our world, no one is going to steal a wallet with $1 billion in it - there is some reasonable assumption that when taking a wallet you are at most stealing a few thousand dollars, and never more money than a person would be comfortable keeping in their wallet. While that's against society's rules for various reasons, it's not a particularly damaging crime. The victim will be perhaps very inconvenienced, but no worse.
However if we lived in a world where a wallet might contain 1 billion dollars, that would be a different story. Now you might very well be causing life altering damage to large numbers of people when you steal a wallet. The decision to do so, knowing the risk, is a much more serious offense. The metaphorical wallet Madoff stole was not only possibly filled with an enormous sum of money, it very likely contained that much. Beyond the much greater and repeated effort that Madoff employed to steal this money than would be needed to snatch a wallet, the very fact he was willing to cause so much potential damage for his personal gain is a much more severe breach of the social contract than a petty thief.
There definitely shouldn't be a simple linear relationship of dollars stolen to days in prison; but that doesn't mean the punishment should be completely agnostic either. These relationships are complex and need to be looked at in context with other relevant factors like pre-meditative effort or degree of remorse. Regardless of one's stance on rehabilitative vs punitive justice, I think we can all agree someone who effortlessly broke core parts of the social contract and would gladly do so again needs to be treated differently from someone who made a bad call in a moment of weakness.
> If you give someone a little shove, they bump their head, and they die
What you're describing is manslaughter or possibly not a crime in most jurisdictions, but your point stands. Generally, murder is an intentional killing, and manslaughter is an accidental killing. But if, say, you give an aggressive drunk a little shove and they bump their head and die, you probably haven't committed a crime. Nonetheless, luck absolutely plays a role in punishment in our current justice system in a thousand different ways. I don't think most people consider the element of luck to be ideal so much as an unfortunate but necessary reality.
With the eggshell skull doctrine, you're talking about paying for damages in a civil case. I think most people see reparations differently than punishment. In civil law, you pay to fix the damage you caused, even though the exact amount comes down to luck. But criminal punishments require some criminal intent. It's a higher bar.
> there is some reasonable assumption that when taking a wallet you are at most stealing a few thousand dollars
I think you're giving thieves too much credit here. They may not expect most wallets to have more than $1000, but I don't think most thieves have some innate goodness in them that makes them want to get a wallet with less than $1000. I think it's the opposite: if thieves knew someone had $1B in their wallet, and the chance of getting caught was the same as stealing any other wallet, I think most thieves would want to steal that wallet more, not less. And I don't think most would care if the money in that wallet rightfully belonged to the investors of Madoff Investment Securities either.
> factors like pre-meditative effort or degree of remorse ... moment of weakness
With these factors, you're judging the thief based on their character. We're both advocating this. The difference is you're arguing someone who steals a larger amount of money has a worse character while I'm arguing they just had greater opportunity.