Hi!
There is the possibility that Joseph and Mary adopted children after Jesus was born. They would be rightly called his brothers and sisters. Nothing specifically states this did not happen. We should not go beyond the Scriptures and assert that Mary gave birth to others - such a statement IS jumping to conclusions. And where the Scriptures are silent, where do we go? Typically, to what the church has spoken on the issue.
It does not help that the Protoevangelion of James was banned by a pope. This is the principal source for the development of Marian theology. From what I have heard, Mary’s role as discussed in patristic literature is nowhere near the role the Catholics and Orthodox place her in today.
Where does this leave us? In the position of needing to be very, very careful in dealing with this emotionally overcharged issue.
-Tina