I would say that you are letting the Puritanical/Calvinistic PoV affect your statement here (whether you realize it or not). Catholics do not look at the body as incidental and immaterial. In fact, our entire sacramental system is based upon the realization that material things do matter…in fact, they help save us. We are not souls trapped in totally depraved bodies (as a Calvinist would say). Our bodies are an essential part of our being. As humans, we are both corporal and spiritual in nature. To deny that the physical plays a part in worship is denying our very nature.
Puritans and strict Calvinists will say that bodies (and anything in the material world) are depraved, and therefore any attempt to bring this physical world in conformity with holiness is vanity because it is unattainable and, therefore, not important to the life of a Christian. They stress the spiritual to the point that the material is a non-issue. The problem is that we are not trying to get away from our bodies, we are trying to sanctify our bodies. When we fast, when we work our bodies in service to the poor, when we adorn our bodies as Temples of the Holy Spirit, we are acknowledging a very Catholic idea: the material world is inextricably bound up in the redemptive work of Christ.
This is the guiding principle behind decorations, sacred music (Gregorian chant, polyphany, etc…), social justice, beautiful architecture, inspiring priestly vestments, etc… You bring the lordship and glory of Jesus Christ to bear on time and space. Not to mention that it is wonderful to look at. The window dressing (cloths etc) brings joy to the beholder and should not create embarrassment for anyone.
So, I’d say that dressing your best is a sign that you have a healthy image of yourself as a body/soul composite being.

While you do not overemphasize the physical to the denial of the spiritual dimension,
denying the physical and saying the spirit is all that matters will ultimately lead to a desolate view of humanity…not to mention boring.