First off, congratulations!!! God bless you all!!! How wonderful!!! I have a 6 month old and a 5 year old, and I remember how hard it was just to do the most normal routine things when your baby is brand new. Personally, I did miss mass when they were newborns for similar reasons. We were told not even to take them out of the house until they were 2 weeks old. To answer your question…I’m sure there are others here who will give you the exact rule regarding missing mass, but if it’s for the sake of your child, I believe it is valid. If it is possible for the two of you to sort of ‘tag team’ to go to mass (one at a time, different masses, while the other stays home with the baby) that might help. However, with the baby being so new, that might make life more complicated.
As far as having mass at home goes, if you can watch a mass on television (EWTN, local etc) like they have for shut-ins, and follow along in your missal, that sounds like a good way to do the next best thing. It’s more effort than a lot of people in similar circumstances would make.
God bless, and congratulations again!!!