Listeria is dangerous to pregnancy, but it is not food, it is a foodborne pathogen. That is why they don't recommend cold cuts from a deli that might have been sitting in the case a while.
Non-pasteurized milk is not recommended during pregnancy because of listeria concerns also.
Food per se is not going to cause a birth defect. Bad stuff in foods that have not been properly handled can sometimes cause problems in pregnancy.
High levels of mercury found in some fish can harm the baby
Not just any food? I really don't know if any will cause a birth defect but some are not good for the baby.