I wouldn’t get defensive as though you need to explain yourself. Just say “yes, we are so happy.” My DIL has 4 boys close together. At first, she didn’t want a big family but she loves having babies and changed her mind. It’s only the couples business but I have heard of women losing friends after they have their 3rd child. Sad really. They weren’t friends in the first place.
Just remember couples without children also get rude comments the other way. I think it is probably more hurtful when couples suffer with infertility.