Sometimes it can be difficult to separate the art from the artist, particularly when the artist in question is especially vile.
I often bring up Death of the Artist, but with books and music I have an especially hard time. Authors, in particular, struggle to keep their works views and politics out of their books; the same is true with musicians, perhaps to a lesser extent.
It does make me worry about the subconscious influences of listening or reading them.