As a religious who regularly wears his habit, I can speak from experience. It varies from Order to Order. Many of the Orders I have encountered, even monastic ones like the Trappists at Gethsemane and Mepkin Abbeys, often do not wear their habits for manual labor. So too, many habited Orders do not wear their habits if it would cause scandal. In many areas of Europe, the poor view habited religious as holding themselves above those whom the religious would help and so the religious do not wear the habit. The habit is an outward sign of an inward conversion. Because of this, some non-cloistered habited orders do not wear the habit while in private community. Everyone within the community already knows of the inward conversion of their brothers and sisters and thus do not need the outward sign. When our Order was founded in 1588, all members kept their habit in a communal cabinet or closet next to the primary door of the convent and only put it on for prayers or when they went out to minister.
Other than this, it is perfectly acceptable to wear the habit to places like the doctor’s office. I’ve worn mine to the grocery store to pick up breakfast for my brothers. Even though I was in South Carolina where there are very few Catholics, I think that was the most pleasant experience in a market I’ve ever had. I got some funny looks initially, but once they realized that I wasn’t actually wearing a dress, they became very happy and welcoming. Watching someone’s face light up upon seeing the habit is something special and uplifting. (Also, the double takes can be quite funny.)