My sister was in a dating situation where the man left his wife, and they just had a baby. Thankfully, my sister ended this horrible relationship, and has moved forward–realizing it was a sinful relationship. That being said–the children in that situation who were younger than yours, lost respect for their dad, all on their own. Kids are wiser and more perceptive than we give them credit for…I don’t believe in bad mouthing a parent to another parent, in these cases–but, trust me–at their ages, they probably have lost respect for their dad, and perhaps on their own–choose to not sleep over. This is a bed he has made with his kids. People think that adultery only impacts the spouse, but it impacts the kids…and when he loses a positive role model relationship with his kids, that will be the price he paid for his behavior. I don’t think you should allow your kids to sleep over…NO WAY!

If he wants to shack up with a woman then he does that of his own accord…not taking the kids into the immoral behavior, as well…introducing them to a lifestyle of sin. Because he’s their dad, does not make it that he can do whatever he chooses with his kids. I would definitely continue praying for reconciliation, but I would start exploring lawyers out there, who can better assist you in this situation. Your husband will play the ‘dad’ card, and the ‘I pay for them so I can see them’ card, but no, I would fight that…sorry charlie but you can’t take the kids into your sinful lifestyle. See you in court over it.
I just think that at some point, you need to stand up for yourself, otherwise your ‘husband’ will walk all over you…no sleep overs…and that’s that. I will keep you and him in my prayers…hopefully, your marriage will reconcile, but you need to give this a time limit in your mind. A true marriage is not one spouse living alone, while the other lives with ‘a girlfriend/boyfriend.’ No, that’s not a marriage…and I am soooo sorry for your pain. I was separated from my husband for a time, years ago, and I know what it does to kids–and our situation was different than this one…but, just know that things will get better, just keep praying for God’s guidance.
(and start talking to some decent lawyers, just in case you need to hire one at short notice)