Anyone using the expression to refer to "the people" or "the press" obviously isn't talking about those groups' legal relationships to the de jure branches of government. My sense (from the Wikipedia article, but also anecdotally) is that "fourth branch" is just a colloquial way to refer to various sociopolitical groups. So I don't see this as a convincing rebuttal to dragonwriter's comment.