Not to lambast you, OP, but to warn you and encourage you:
*"Likewise, you younger members, be subject to the presbyters. And all of you, clothe yourselves with humility in your dealings with one another, for: “God opposes the proud but bestows favor on the humble.” So humble yourselves under the mighty hand of God, that he may exalt you in due time. Cast all your worries upon him because he cares for you.
Be sober and vigilant. Your opponent the devil is prowling around like a roaring lion looking for [someone] to devour. Resist him, steadfast in faith, knowing that your fellow believers throughout the world undergo the same sufferings. The God of all grace who called you to his eternal glory through Christ [Jesus] will himself restore, confirm, strengthen, and establish you after you have suffered a little. To him be dominion forever. Amen*. 1 Peter 5:6-11
I had a sailing instructor once who warned that none of us had better ask him if it was OK to capsize the boat. Whether the boat would be OK or not was besides the point. A sailor never takes the chance that he’s going to capsize his boat. Period.
You in no way had intentions of infidelity, but you’re not displaying a burning desire to remain faithful, either. It is not a sin per se, but it is a serious temptation to want the attention of sexual attraction from those whom you are not free to marry. You are walking where you are vulnerable to attack. Forget about mortal vs. venial in this case. Work on taking every precaution so that you never ever capsize the boat. The difference between mortal and venial comes into play when you’re deciding whether you need to confess before or after the next time you receive the Eucharist, and not the difference between sins that grease your way to ruin and those that don’t. There is not a safe level of sin. Don’t even think that way.
Renounce your desire to indulge your own vanity and go back to courting your wife. Pride is a deadly sin. Presumption is a deadly sin. Taking your marriage for granted puts it in danger of being capsized. Cherishing and nourishing it, OTOH, helps to guard against temptations that would ruin it. Trust in God, but row away from the rocks!!
Let us now picture a man with a crucifix around his neck and a white depression on his finger where his wedding ring was ten minutes ago. Is this a witness for the Gospel? If you wouldn’t do it while wearing a T-shirt that says in large letters, “I’m a Catholic from St. Pius X Parish”, then don’t do it at all.