The Liturgical colors of vestments have symbolic meanings that correspond to their respective Liturgical seasons.
Green- ordinary time
Purple- Advent and lent
Red- Pentecost/feasts for martyrs
White/Gold- Christmas and Easter
Black- All Souls’ Day and funerals (traditionally)
Rose- Laudare and Gaudete Sunday
White/Gold can technically be used on most days of the year as a substitute and is more often used for funerals nowadays. Additionally, several places have permission to use Blue on certain Marian feast days, but this is not common. Other collors like orange, yellow, and silver have no Liturgical usage, as far as I’m aware.
So in general, the Liturgical colors are meant to follow the Liturgical year (with some exceptions). As far as the “whore of Babylon” thing, I’ve never heard that argument in relation to Liturgical colors.