It's the same for all the non-EU French territories. EU citizenship is not territorial, it's directly linked to the person's nationality so you wouldn't remove EU citizenship from a French citizen just because he happens to live in New Caledonia. French or British citizens living in Canada also are EU citizens after all.

The state of British citizenship is a bit more complicated though I think. A bit like US citizenship which kinda depends on which US territory one lives, as far as I understand.