Peter Elliott (who was recently ordained as a bishop in Melbourne, Australia) argued along these lines:
"… While the alb may seem visually appropriate for a female server, in my opinion, the cassock and surplice should never be worn by a girl or a woman. The cassock and surplice is choir dress for male clerics, extended by custom to young male servers, and this custom is rejected by various liturgists since boys are not clerics. But choir dress conjures up a mlae “clerical image”, so a female wearing male robes could be described as a bizarre ecclesiatical form of “cross-dressing’. …”
(Peter Elliott, Liturgical Question Box, Ignatius Press, 1998, ISBN 0-89870-677-7, page 61).