I will include the footnotes in what I quote below.
Taken from
Beginning Apologetics 5: How to Answer Tough Moral Questions
41 Father Thomas J. O’Donnell, SJ points out that all authentic and approved Catholic theologians have taught that barrier methods such as condoms, diaphrams, and spermicidal jellies and foams pervert the marital act itself. See Medicine and Christian Morality (New York, NY: Alba House, 1996), 250-251.
It should be noted that this section (above) taken from that booklet is not official Church teaching on this topic, but the opinion of whoever wrote the booklet. It is not an opinion that is universally held nor binding.
To the OP, please listen to your priest and work with your wife to find an agreeable solution. You cannot use condoms, even to prevent what you see as a greater evil. It is a founding principle in Catholic morality that we may not do evil that good will come of it. This rules out using a condom, even for the sake of preventing failed implanting of fertilised eggs and/or maintaining your marriage.
It should be noted that failed implanting of fertilised eggs and other spontaneous abortions happen all the time. They are a natural evil, and only a moral evil when intended. Some might say they are intended if you choose contraception that increases their likelihood, but the jury is out on whether the pill, or any other particular contraception, actually does this. It’s something of a grey area, both medically and thus morally.
If you and your wife have identified that only barrier methods are suitable, you could suggest to her that if she insists on continuing to use contraception then she should find a method that she can administer without your involvement.
I know you are in a very difficult situation here. The clear moral answer is that you can’t yourself use condoms; this is hard, but faith is never promised to be easy. The less clear answer is what you and your wife do having ruled that out.