Deputy ministers are supposed to be the top non-political/bureaucrat position in a ministry, so theoretically it wouldn't matter if the government is conservative. The elected minister sets the goals and gives general directives, the professional deputy is in charge of execution. In reality this neat distinction is often broken, some governments politicize appointments even below the minister level, etc.