Let’s deal with these issues one-by-one.
First, if by “For 4-5 years, I have been leading a single gay man life,” you don’t mean you’ve been sexually active outside of marriage, what on earth do you mean? It sounds as if you’ve been leading the life of a bachelor who doesn’t date women, or perhaps one who only socializes with gay men. Do you mean that you’ve only been looking to ogle other men? Do you mean that you have engaged in homosexual actions, but none that have been genital in nature? I honestly don’t know what you mean.
Second, I would strongly advise you that if the love of your life is what you’d call “not pretty,” that you leave her looks out of her description. Certainly don’t put it first. That betrays a shallowness that does not become you. (Not unusual in someone brought up in a looks-are-everything culture, but just don’t do that.)
Third, let’s get rid of this “the same-sex attraction is still there in my body.” I’m not looking for a quarrel about whether that is literally true. I don’t have any trouble accepting that a compulsion to do something could be organic. Still, if someone were to go out and get drunk once and feel the “click” and then stayed away from alcohol thereafter because he feared he could never have a controlled relationship with alcohol, but he hung around in bars, including in his social circle people who couldn’t control their drinking, I’m not sure he’d be considered an alcoholic. I think he’d be considered someone at risk of becoming alcoholic, and not handling it very prudently. IOW, if you are tempted to homosexual acts and want to keep with the teaching of the Church, the very last thing you ought to be doing is hanging around with people who find those acts right and necessary to their happiness. (What else you could mean by “living” the life of a gay man, I do not know.)
Likewise, it has already been noted that a great many people are tempted away from chastity, the heterosexual no less than the homosexual. Heterosexual men are afraid to make a commitment to a woman for just the reasons you describe. Again, I don’t doubt that some men have stronger physical urges than others, but that does not mean you are a body let loose on the world with free will just along for the ride. The free will is a man riding an elephant. Mastery is not possible by brute force, but compliance of body to will certainly is possible for those who are wise, patient, and firm. The man who does not have charge of his own sexual faculty, or who looses charge “only once in awhile,” is most surely a man riding a huge and unpredictable beast of some kind! Yet you seem someone who has always had control of the urges in your body. I would not worry that you will be overwhelmed by compulsion. You need to switch your noodle if you don’t want to be seduced by your desires, but you have self-mastery enough to stay chaste for a lifetime, if you’re willing to use it.
Should you tell her? Yes, you should. That doesn’t mean you ought to self-identify as homosexual (because you also have attractions for women) but that you absolutely should tell her if you’ve had attractions for men more often than for women. Still, if you have really kept chaste for your entire life, that is a point in your favor. Wow, is that a point in your favor. You have kept yourself pure in order to please God, not because some other person would be jealous if you did. In other words, this wasn’t “you stay faithful to me, and I’ll stay faithful to you.” You have developed the real gift of continence, and that is what is needed.
BTW, if by “not pretty”, you mean that she arouses you *far less *than most other people in the general population, to the point that she really doesn’t set you on fire at all, think hard about that. A married man owes a marital debt, and the norm is that he will give his wife what she ought to have from him in great joy. It is a debt, but it is like the debt to love and honor your parents. It ought to be fulfilled naturally and willingly, happily; excepting the days when you are just being self-centered or are physically too tired or sick or something of that nature, it ought not be a burden. That doesn’t mean it never is, but if it does not seem something that would bring you great joy, you might find yourself hurting her feelings in a very deep way. Think about that. Pray about that. Get some advice about that.