Stress has hormones associated with it. It's those hormones that affect the baby. If you exercise, eat well, and do things to help reduce stress - prayer, deep breathing, massage etc you can help buffer the effects. So it is bad, but if handled smartly you can avoid preterm labor and other negative effects.
Your baby is fine! It's difficult to be more stressed than I was on my first trimester! My dad passed away when I found out I was pregnant. I inherited his three business and was really stressful. Plus, the employees decided to sabotage my work and was really tough time.
However, I began having mini contractions due to stress and doc told I had to relax! Fortunately, my hubby helped my out a lot and I managed to stay home and work from home 4 days a week, and go to the office 1 day.
So, stress is not good, but as long as you're not feeling anything different or strange, you'll be fine!
Stress isn't good for you or the the baby. At least that's what they tell me all the time! I'm always stressed, but I think the little ones will be fine. Just try to relax as much as you can.