It can cause depression and irritability from what I've read. Just went to see a Naturopath and she agreed with some advice I had received at a health food store that I should be taking 4,000 iu of Vitamin D3 daily. I had my Vitamin D level checked in January and it was 36. My Naturopath said she'd like it to be about 60. I've read that most people who live north of the Carolinas are probably Vitamin D deficient. Not sure about whether a deficiency would cause anxiety though. Have you had the blood work done to see whether you are deficient?
See "Other Symptoms" at:
http://www.ehow.com/facts_5127269_signs-symptoms-vitamin-deficiency.html