I've never heard that. I only heard that if you exercise during your pregnancy you have an easier labor/delivery and that it's easier to get the weight off after. I think if you're taking your prenatals you'll be fine. Baby needs the folic acid for brain development.
Don't worry too much. Just do the best you can.
Take a walk on your lunch break, too. Even just a few times a week. It's really good for you. There are lots of chair exercises/ stretches that you can do without leaving your desk, without even really stopping what you're doing. Little changes to your routine can make a big difference! Getting your blood flowing, raising your heart rate even just a little bit, will help your baby out and help you too. Good luck!