It's been an emotionally joyous time for the couple after they welcomed their first child into the world last month