It could have to do with distance, eg. Mary wouldn’t of had an extremely close day-to-day relationship with her cousin, Elizabeth, wheras those who hung around Jesus would be very close
That could never be a reason for the term in a Semitic manner of speach: blood is thicker than location in all the Old Testament.
But they were not actually cousins, as far as I can tell, but kindred in the sense of being from brother tribes of Israel, Judah and Levi. John the Baptist was the descendant of Aaron the brother of Moses.
However, this doesn’t remove the many Biblical and earliest Patristic difficulties with saying that “brother” was used in the Semitic manner in the Greek of the New Testament. JESUS must have had many cousins, not just a hand-full. Why, we must ask, was it understood that only two or three girls were HIS sisters, and only four guys HIS brothers? Why did the idea that they were cousins come long after both the idea of brethren through Mary and and the idea of step brethren through Joseph?
I have watched person after person debate from their different points of view, and have seen early Fathers misquoted, and various people with the same common name merged into one hybrid character (which then sets up the scene for the DaVinci Code nonsense, that claims no Saint got his peers’ identities right).
JESUS and John the Baptist, though, were likely not even second cousins. But where Scripture speaks of Barnabas and Mark, it calls them cousins, not brothers.
As for the brothers and sisters of JESUS, we do have to give a better reason why their parents are not named with them, but they are named in connection with JESUS and HIS mother and her husband.
It would be interesting to see what the earliest known quotation in support of each of three theories is, and see if it is known who said it, and in what context, etc.