Blender had moderate success when it was closed source, but not enough to pay its development, so it was going to die.
After its creator raised €100,000 to release it under the GPL, Blender became the leading open-source 3D tool it is today.
And they make enough money from recurring donations, service subscriptions, merchandise, conferences and trainings.
Blender is great, I also use it. It's a nice example on how a non-developer tool is successful with open source.
There are/were plenty of open source RPG makers, but they never gained any real traction.
I considered open sourcing my product in the past (did so with a previous game), so maybe one day. I still have some big things planned :).
I think Blender was (and still is) exceptionally good at community building. Just freeing your product might not get you enough traction by itself.