One thing people miss is that bugs in open source are much much easier to fix when you catch them right away. You find more bugs when you test aggressively, but the effort per bug is usually significantly lower.

I think the key is to differentiate testing from deployment: you don't need to run bleeding edge everywhere to find bugs and contribute. Even running nightly releases on one production instance will surface real problems.