At Bryan Caplan’s recommendation, I have been reading Michael Huemer’s book The Problem of Political Authority. Michael tells this parable on pages 142-143:

Marvin is in need of food, without which he will suffer from malnutrition or starvation. He plans to travel to a nearby marketplace, where he will be able to trade for food. But before he can reach the marketplace, he is accosted by Sam, who does not want Marvin to trade in the marketplace, for two reasons. First, Sam’s daughter is going to be shopping in the marketplace, and Sam fears that Marvin might bid up the price of food. Some vendors might even run out of bread if too many people come to the marketplace. Second, Marvin comes from a different culture from most people presently at the marketplace, and Sam fears that Marvin might influence other people and thus alter the culture of the marketplace. Sam decides to solve the problem by force. He points his gun at Marvin and orders Marvin to turn around. The starving Marvin is thus forced to return home empty-handed. 

Sam’s reasons for coercing Marvin in this story are clearly inadequate. Furthermore, Sam will be culpable for whatever harms Marvin suffers as a result of being unable to reach the marketplace; they will be harms that Sam inflicted upon Marvin. If Marvin starves to death, then Sam will have killed him. This is true even though Sam was not responsible for Marvin’s initial situation of being hungry and out of food; it is true because Sam actively prevented Marvin from obtaining more food. If a person is starving, and you refuse to give him food, then you allow him to starve. But if you take the extra step of coercively interfering with his obtaining food from someone else, then you do no merely allow him to starve; you starve him. The same point applies to lesser harms: If, for example, Marvin merely suffers malnutrition as a result of being unable to reach the marketplace, Sam will have inflicted this harm upon him. 

The behavior of Sam in the story is analogous to that of the government of any modern country that excludes poor immigrants. Potential immigrants from developing nations come to participate in the marketplaces of wealthier countries. The governments of the wealthier countries routinely forcibly exclude these potential immigrants. As a result, many suffer greatly diminished life prospects. The government does not merely allow harms to befall these would-be immigrants. If the government merely stood by passively and refused to give aid to potential immigrants, then it would be allowing harms to occur. But it does not stand by passively; the government of every wealthy country in the world deliberately hires armed guards to forcibly exclude or expel unwanted persons. This coercive intervention constitutes an active infliction of harm upon them, just as Sam inflicts harm on Marvin in the story above. 

The most common reasons given for immigration restrictions are twofold. First, that new immigrants compete with existing Americans in the labor market, thus driving down wages for unskilled labor and making it more difficult for American workers to find jobs. Second, that if too many immigrants enter the country, they will alter the country’s culture. The first concern is analogous to Sam’s concern about Marvin’s competing with Sam’s daughter in the marketplace. It is not permissible to use force against another person simply to prevent a third party from suffering economic disadvantage through normal marketplace competition. The second concern is analogous to Sam’s concern about the culture of the marketplace. It is not permissible to use force against another person simply to prevent that person from influencing the culture of one’s society in undesired ways.