And teeechnically, Roman women didn’t wear veils except on their wedding day (the rectangular red flammeum, which means “flame colored”, and which was wrapped around the head but left the face uncovered).
A married Roman woman wore a palla (“cloak” or “shawl”), which was pretty much an outer garment, pulled over the head to make a hood for the shawl. (The men’s version was the “pallium,” which was a big woolly philosopher’s cloak. A bishop’s pallium is a tiny baby version.) Indoors, a woman would usually just wear the palla across her shoulders as a shawl. (So St. Paul was basically telling women to behave like they were in public, keeping the shawl up, and not like they were dining privately at a friend’s house, with the shawl down.)
The original palla for women would have been good Roman wool, but they got gauzier and gauzier for the rich and fashionable. In the West toward the end of the Roman Empire, the palla continued to be worn for a very long time, and eventually turned into all sorts of mantles, shawls, hoods, headrails, etc., in the Middle Ages.
Byzantine women also started out wearing the palla; it got a little more elaborate. But noblewomen began to cover their heads with a headdress that let them put up their hair but also show it off, and which could have diadems balanced on top. (This also showed off the elaborate necklaces and collar embroidery of the day.) Since noblewomen could not wear the paludamentum worn by the empress, they began to go for semi-circular cloaks instead of the palla, or to wear the palla connected to their poofy snood/headroll hat. Ordinary women tended to stay with the two stolas (long sleeve and short sleeve) and a palla.
There’s a lot to be said about the history of Christian female fashion. But the main thing to be said is this: It varied. A lot. If you think something is typical of the Byzantine Church today, it’s probably typical of Greece after the Muslims had reigned there for a couple of hundred years. Anything from Southern Europe worn in summer is going to tend to be cooler than anything for Northern Europe worn in winter.
The main thing with hats is that they tended to be cooler and more mobile. So you had times when people wanted the dignity or sun protection of big hangy things on their heads, and other times when they really wanted something that would stay out of their face and their way.