I’d like to push back a little at the pooh-poohing of the OP’s anxiety. It genuinely is a major responsibility to conceive child and there are considerable risks to both child and mother, and it’s not bad that he understands that. None of us here will be there to help you if your wife has to go on bedrest for a full trimester or two, your baby has a long NICU stay, your wife suffers repeated miscarriages, or any number of fairly common complications.
On the other hand, not having a second child also comes with substantial risks. While the risk of chromosomal abnormality at birth when the mother is 37 is 1 in 125, the risk that an only child will face sole responsibility for elderly parents is 1 in 1.
Best wishes and I hope that whatever you choose to do will be the right choice.