I agree that you can’t lump all fantasy together, and you have to know yourself and know how things will affect you. If a fantasy or sci-fi novel introduces a morality or worldview that is directly contrary to the truth, are you the kind of person who will let that wrong thinking in? Sometimes, I read things that I know are wrong or untrue for the purpose of strengthening my faith by refuting the untruth. There is value, too, in recognizing the literary quality of something while also recognizing the theological/moral errors in it. If you aren’t sure of what you believe, or if you aren’t sure you can defend it against an onslaught of error, then it’s better to be cautious.
Really, though, I can’t imagine anyone apart from a few nuts discussed in another thread (who believe C.S. Lewis was a sun-worshipper) who would object to the fantasy and sci-fi writings of noted Christians like J.R.R. Tolkien and C.S. Lewis. I’ve found that their fiction has always brought me closer to God. Even though I don’t know the religious leanings of some other authors, like J.K. Rowling, I’ve found that there’s plenty in their books that reinforces the truths found in Christianity. There are also authors like Philip Pullman, and to a lesser extent Robert Jordan (just my opinion on Jordan), whose books tend to contradict Christian truths or values.