Maybe because most of them were born there and also their parents and grandparents?

That is called birthright and the way I see it, it applies to both groups. And the conflict will never be solved (without large scale genocide), if both groups largely negate the other groups rights.