I was thinking the same thing, it's really odd that a dr. told you something like that. How could he possibly know that was the exact reason. True, stress isn't good for your baby but dont let it make you feel like it was your fault. Sometimes these things just happen and it is out of our control.
I find it odd that a dctor would say this to you, Most of the time a miscarriage or stillbirth is caused by reasons that cannot be determined.
stress can cause pre term labor and if you go into labor before the baby is viable s/he can be a stillborn or pass shortly after birth. you can request an autopsy (which is when they look at the baby internally to find out if there were any other factors relating his/her death) and find out more information.
i am so sorry that you experienced this loss. my thoughts and prayers are with you.