A way to think about it is, if you went into the liars house to prove him wrong, you could find one non-green hat to prove him wrong. If he has no hats you wouldn't be able to prove him wrong.

Which reminds me of a quote from the British TV series Yes Minister: "A good speech isn't one where we can prove he's telling the truth. It's one in which nobody else can prove he's lying!"