My understanding is that a Muslim woman is forbidden from marrying non-Muslim man, because in the Muslim understanding the man is the guardian and and “trustee” of the woman. It is not permitted for a non-Muslim to be the “trustee” of Muslim, because a non-Muslim theist will not permit his Muslim wife to practice Islam. In contrast (by Muslim teaching of at least some imams, I do not mean by Catholic teaching), the Muslim husband is not permitted to prevent a Christian or Jewish wife from going to the synagogue or church to practice her religion. He is bound to let her practice her beliefs, if she believes in the divine teachings that were the forerunners of Islam.
Shaykh Dr. Yusuf al-Qaradawi wrote:
*"The Muslim woman is not given in marriage except to a Muslim, according to the saying of the Most High: “Wed not idolatresses till they believe,” and then He says “…and give not your daughters in marriage to idolaters till they believe…” [Qur’an 2:221]. Then He made an exception, saying, “The food of the People of the Book is lawful unto you and yours is lawful unto them. (Lawful unto you in marriage) are (not only) chaste women who are believers, but chaste women among the People of the Book” [Qur’an 5:5]. Thus it is permitted to marry chaste women among the People of the Book, but it is not permissible for men (among the People of the Book) to marry Muslim women.
"This wisdom is this: The Muslim believes in all the messengers, including Moses and Jesus (peace be upon them). (He also believes in) all the books, including the Torah and the Gospel. On the other hand, the People of the Book only believe in their messengers and their books. Islam has permitted for the wife from the People of the Book who is married to a Muslim to go to her places of worship, such as the church or temple. On the other hand, the People of the Book do not permit the Muslim wife–if they had married her–to go to the mosque and manifest the rituals of Islam. *
and from al-Azhar:
*
"It is true that Islam permits the Muslim man to marry a non-Muslim woman (Christian or Jewish), and does not permit a Muslim woman to marry a non-Muslim man. At first glance, this might seem like inequality. But if one understands the true reason for this, one ceases to wonder, and ceases to imagine that it is inequality. For Islam has a point of view on this which clarifies the wisdom behind it. All laws in Islam are based on wisdom and true benefit for all parties.
"Marriage in Islam is based on “love and mercy” and cohabitation. Islam seeks to build the family on sure foundations, which are guaranteed to perpetuate the marital relationship. Islam is also a religion which respects all of the previous divine religions, and makes faith in all of the previous prophets part and parcel of the Islamic creed. Therefore if a Muslim man is married to a Christian or Jewish woman, he is commanded to respect her creed. He is not permitted–from the Islamic point of view–to prevent her from practicing the rites of her religion, and going for that reason to her church or temple. *
That is what Islam teaches, according to my understanding. As far as what he is doing with the wedding ring he pledged to wear when he married you, this may be something to look into. It is likely you could not inquire on your own, but perhaps your pastor can inquire with his imam? Then you could talk to your pastor about what your situation is?