You also have to remember that at time they still mainly thought of family in extended terms, not nuclear, back then. (The process of the breakdown of the family from an extended unit into nuclear units - caused by the population booming and the Jewish custom of dividing the family land - was just starting to occur.)
I’m just gonna elaborate on this a bit.
As per Jewish inheritance laws, the eldest son inherited the family home and a double share of the land, while the rest was divided among the other sons. (Deuteronomy 21:15-17) Dividing the property was fine in theory, but as you go further down the family line, the land that will be inherited will get smaller and smaller until the individual plots become too tiny to support even one person - in other words, pretty much worthless. In those cases, the younger siblings had no other choice but to sell their land for cash and consider other options: work someplace else, rent, beg, steal.
The other way was for brothers to (continue to) live together and keep a smallholding undivided as the joint property of the ‘father’s house’, rather than divide it. (cf. Psalm 133:1 “How good and pleasant it is that brothers dwell together in unity” - the term “dwell together” there is actually the legal term for joint tenancy.)
The two grandsons of Jude, one of Jesus’ ‘brothers’, Zoker/Zechariah and James, were said to have been farmers who jointly owned thirty-nine
plethra (depending on how you interpret
plethra, it could either be something as small as nine to ten acres or at the maximum, around twenty-five acres) of land with which they supported themselves. Of course, Jesus and His relatives were not wealthy people, but IMHO the fact that these grandsons still owned some land show that Jesus’ family were not exactly destitute either. Two generations back, the farm that they owned would have probably been owned by Joseph and his brother Clopas as well.
That being said, thirty-nine
plethra of land was not much to support two families. In fact, that might explain Joseph (and Jesus) became artisans/‘carpenters’ (
tekton): not as an alternative to working the land, but to supplement the family income because the land they tilled was not enough.