I can understand your problem. Once we like a certain type of music it remains a part of us - especially if it has been a part of our life at some times.
I have a particular weakness for The Rolling Stones and AC/DC. I enjoy the music, the energy and the show. I always enjoy anything of theirs which I catch on youtube.
However, in the room with my computer, right behind where I sit, is an image of the Mary, with the Immaculate Heart. If I’m watching, say “Let’s Spend the Night Together” (Rolling Stones) or “You Shook Me all Night Long” (AC/DC) I might ask myself, “What is Mary thinking?”. And then I turn it off.
With time (a long… time) my music tastes have changed. At first, I would force myself to avoid the worst of Rolling Stones and AC/DC, but with time I’ve found other music, and lost almost all interest.
There is still a problem that some good, strong songs, have one or two questionable lyrics. I usually go for the whole song in this case.