I know people that work as contractors for the Dutch government. The government doesn't save money by hiring them through contractors. They cost more through contractors. But contracting allows private companies to act as gatekeepers and pocket some cash for essentially supplying full time employees. It's a form of corruption by well connected private contracting companies.
I think a large part of the reason is that government hiring is rather permanent. It's often prohibitively expensive/hard to get rid of underperforming or superfluous employees. Contracting is a way around that. That allows hiring workers in a temporary (project) budget. For decades, sometimes.