OP - yes, it can be very, very difficult.
My wife and I used to contracept, before we married and while we were married. Now we (I) am trying to be the best Catholic that I can, I have been for several years now, and we’ve been using NFP. We have read every book and contacted every authority on the matter we can think of - and it doesn’t work for us. At all. Condoms worked to avoid pregnancy 100% of the time for us, NFP ZERO percent (calculating percentages based on observed pregnancy per year practicing). Also, we have sex maybe once day per month. My wife’s body does not work as others’ do apparently, and since our last 3 NFP failures, she is unwilling to take any risks.
Our sex life when we were contracepting was better. Our marriage was better when our sex life was better.
I do not have a low sex drive. I struggle with sexuual sins, namely lust and pornography, and following the Church’s teachings on contraception has not only been a great strain on our relationship, it’s resulted in my cross getting heavier still. Truth be told, it all gets me quite irked. The Church says:
Better to marry than burn with desire. So I marry. The Church says Don’t contracept. so I don’t, and good-faith NFP efforts fail us, and we have 4 kids in under 5 years - and I’m now effctively celibate, but still burning with the same desire as before I was married, only now my wife and I are overburdoned with more children than we can readily handle and she pretty much resents me for encouraging her to learn and use NFP instead of contraception.
To be honest, I’d hit a rest button if given a choice. and being that I’m not given that choice, I think I’d probably off myself if I didn’t have children depending on me and a God that’d likely let me wallow in a lake of fire for eternity if I did.
I wish that the Church was more honest - and I think it did used to be more honest. “Follow these rules or be tortured. following the rules is a sort of torture, but hey, at least it isn’t permanent. Crosses aren’t light and the gate isn’t wide.” The “Good News”, it seems to me, is that there’s an out at the end and that it will all stop sucking after we die. I hear some speak of the happiness that comes wioth keeping God’s commands. Hasn’t worked that way for me. I’m miserable (due to guilt and the knowledge that I have no reason to be expecting heaven when I die) if I sin and I am miserable when I mannage to keep His commands. I tend to think of my sexual appetite as a thorn, not a gift. No joy has really come from it - only guilt and angst. The one bight side, if you can call it that, is that messing up and falling into sin gives me a reason to come crawling back to Confession.