Also consider the insecurity that our system creates for people. Losing a job can snowball into something much larger. There goes your healthcare, mortgage payment, retirement contributions, ability to find another job (depending on the current state of the market), etc.
That's assuming your full-time job is secure. I think many of us have lived through rounds of layoffs.
Some of that insecurity is self-inflicted
If your mortgage payment and overall living expenses are so high that you can only get by with a high paying job then you've shackled yourself to your org and should fear being fired
If you live well below your means you'll have a large savings buffer and won't care so much
> The philosopher Diogenes was eating bread and lentils for supper. He was seen by the philosopher Aristippus, who lived comfortably by flattering the king. Said Aristippus, "If you would learn to be subservient to the king you would not have to live on lentils."
> Said Diogenes, "Learn to live on lentils and you will not have to be subservient to the king."