You are couching your argument in a definition - namely that refugees must be moving across an international boundary - so the Holy Family must be ‘displaced persons’. You are quite literally arguing the semantics, which, isn’t the point at all. The point is people in trouble, in desperate, dire trouble, who cross distances to end up somewhere their family can exist and live safely.
But let’s talk semantics anyhow!
The kingdom border - Judea and Egypt, is in fact, international boundaries. You see, empires by definition are not nations. They are not nation-states. They are empires - a super-territory made up of many nation-states and regions. This is an important distinction in the history of geopolitics, the difference between a nation and an empire. So the Holy Family indeed did not cross any imperial boundaries. But they did cross the boundary of the kingdom of Judea (a nation-state) into the kingdom of Egypt (another nation-state), thus indeed crossing an international boundary. Ergo, by your definition, the Holy Family were refugees.
Which again, isn’t the point what they were called. The point is that they were people fleeing to a foreign land, with foreign laws and foreign customs and foreign people, for the safety of their family. And I have to wonder very much about how we are treating these foreigners in our land - Are we letting brotherly love continue? Or are we neglecting to show hospitality to strangers, thereby mistreating angels unawares.