Dear friend,
Such behavior is a mortal sin. It is not a sin of malice, but of weakness. You need not take such behavior personally. Men are wired differently than women and have a much more difficult time avoiding this sin. Certainly, this does not warrant your leaving him. It does warrant his going to Confession, though. Pray for him as will I.
Fr. Vincent Serpa, O.P.