See Liturgical Vestments;
newadvent.org/cathen/15388a.htm
Personally; I think if a priest is faithful to his sacramental ordination and what it signifies to him in between his service to God first in union with the Church especially in regard to sacramental clerical duties that happen within the walls of the Church; the priest should not be haphazard in the way he presents his vestment appearance in front of the congregation. Most priests when called out to confer the sacrament of the sick on the faithful in hospitals and homes bring a stole with holy oils and water to the bedside of the Infirmed. The Stole represents the specific mark of office, being the badge of the diaconal and priestly orders. Many priests from pre-Vatican II would kiss their stole as a significant reminder of their priestly ordination and service duty to God. When placing their Stole about their neck in the sacristy they would say a private prayer: ("Lord give me this last day the garment of immortality that was forfeited by our first sinful parents “Adam and Eve”. I see many priests today especially those coming out of modern seminaries who have this nonchalant attitude as to whether they should say Mass or do Baptisms, Weddings, Funerals and the sacrament of Penance with half vestments on or off.
Is this important ? Yes it certainly is. A priest is first and above all a man of God and should wear all his priestly vestments humbly and faithfully in Gods service.
A priest is not a people pleaser. Should a priest wear his stole when conferring the sacrament of confession ? Definitively Yes.