I’m afraid that I’m confused as to how you knew the altar server was wearing a T-shirt and shorts under his robe. If you happened to see him vesting in the sacristy, then what he wore under the robe is not your concern, but may very well have been chosen because it is summer and he would be wearing a robe on top of his street clothes. It would have been good for him to also wear shoes, not sandals, but I can’t see making a formal complaint over that.
Now, if he was not wearing the ordinary robe of the altar servers over his street clothes, then you could certainly ask the pastor—as a matter of curiosity, not hostility—why the altar servers did not wear their proper robes. Be prepared for him to say something like, “Well, it’s summer, you see…”
Personally, I would let it go and save your bullets for complaining about a truly significant liturgical abuse (e.g., one that compromises the validity of the Eucharist).