From the document Redemptionis Sacramentum:
"'The vestment proper to the Priest celebrant at Mass, and in other sacred actions directly connected with Mass unless otherwise indicated, is the chasuble, worn over the alb and stole. Likewise the Priest, in putting on the chasuble according to the rubrics, is not to omit the stole. All Ordinaries should be vigilant in order that all usage to the contrary be eradicated. (122-123, also see the GRIM #337)
Since the instruction states that it is the ordinaries responsabilit to ensure that the stole and chasuble are worn correctly, you are well with in your right to notify in writing the local bishop, if after you approch the pastor he fails to ammend his ways.
Pax Domini,
Casey the Seminarian