I think there's a sort of divide-by-zero problem here. Does an empty set of hats have a color? You could arbitrarily define "all my hats are green" for the empty set as either true or false as part of a consistent logical system. There isn't enough information in the question to know whether we should pick one or the other, though there's probably a colloquial preference for true.
Is there not a convention within formal logic defining this type of statement to be true or false?
If set of his hats is the empty set then it contains no hat which is not green, which would make his statement true. As siblings have said, this is a “vacuous truth” in formal logic.
Edit to add: if you find this problematic consider that the statement “All my hats are green” in formal logic is identically equivalent to
For all hats h in my hats, h is green.
So for this statement to be false there needs to be a hat in the set “my hats” which does not have the property that it is green. If “my hats” is empty or indeed if somehow contrary to the rules of the game “my hats” only contains things which are not hats or all the hats it contains are green then the statement is true.
Since the speaker is a liar the statement cannot be true. Therefore there is at least one hat in “my hats” which does not have the property that it is green.
The maths students will be learning this in the context of negation and will have learned that the negation of a universal (“for all”) statement in predicate logic is an existential (“there exists”) statement. Since we have a liar we have to negate what the liar says so since the liar says
“For all hats h in my hats, h is green.”
We negate this and deduce
“There exists at least one hat h in my hats such that h is not green.”
In the context of old-fashioned predicate logic this is not ambiguous.