Because once they come in sufficient numbers they will turn your country into the country they fled from - and then you are in trouble.
Which river is it in Ireland that they dye green every year for St Patrick’s day like they do in Chicago?
So for the United States, it would primarily be family-oriented Spanish speaking Catholics whose kids will be bilingual and grandkids will speak only English? There have been waves of immigrants before where the Irish or Italians or Germans were seen as "invaders" undermining the character of the country. And then their descendants fully integrated and became part of the culture.
Also the US and Western European countries are in much better economic and civic conditions that the immigrants can take advantage of to live better lives and contribute.
This is a slippery slope argument at best and jingoist rhetoric at worst.