Find someone (friend, family) to take care of the baby for a couple of hours. Take her out - movie, go-carts, coffee, shopping, something adult. Allow her to tell you not just what she thinks about being home with the baby, but how she feels.
You can do the same thing if you don’t want to leave the baby. Figure out a time the baby sleeps (great part is that they do this a lot). Grab take-out and put it on your good plates and light a couple of candles. Maybe rent a movie she wants to see. If the baby fusses, you take care as much as you can.
Are there any social groups she can get involved with? My wife and I were in the same issue with our 1st. Joining the Moms group at Church was a big help - got her hooked up with other stay-at-home moms for moral and logistical support.
Oh, and hey, congratulations on the baby. You sound like a great husband and I bet you will make a good dad.