The primary reason I have come across is misinformation. There are some virulent and hateful tracts out there that seek to undermine the authority of the Catholic Church, or even to proclaim it as un-Christian or pagan. All of this is due to misinformation. It is rarely, if ever, the individual spreading the information’s fault, nor their pastor’s. Rather, their instruction comes usually from some disillusioned but charismatic individual who could not see past their own personal opinions and taught said opinions to their followers, who taught their followers, etc. Brought up in a fundamentalist Southern Baptist household, with United Methodist influences, I was taught to believe many lies about the Church, as was my family, some of which were properly instructed when my mom married a Catholic man from Italy. Others remained hostile to the Church even as I began my investigation. By my conversion this Easter, fortunately, they were in full support.
Some of those who are anti-Catholic will argue that they are not misinformed, but just a basic knowledge gleaned from participation in the Mass will reveal otherwise when they seek to prove it, even more so with knowledge from RCIA instruction. Unfortunately, there are those who, no matter how much support you have, they will not believe you, and will begin to recycle their arguments in order to avoid admitting their own error.
This is just what I have experienced, it is not necessarily all cases, but it is what I have had the unfortunate displeasure of experiencing. Hope this helped.