There are other people who can be within the territorial boundary of the US but not subject to its jurisdiction. An invading army, for example. It leaves some interpretation of this question but it does not leave unlimited interpretation to the President, or even to Congress. That is the understanding of Wong Kim Ark, which overturned the law Congress passed 14 years after the ratification of the 14th amendment.