Have your serum calcium (part of CBC lab work), ionized calcium (not part of CBC), and PARAthyoid checked. Low vit d is often a symptom of hyperPARAthyroidism. Even if your serum calcium is "normal" (mistakenly assumed by most drs means no disease) you can still have a high PTH level which means you HAVE parathyroid disease. And if you have it, it would explain your anxiety as these type of symptoms are common. Check out info at www.parathyroid.com. This website is hosted by the TOP guy on PARAthyroid disorders in the country. It very comprehensive and written at a level the average person can understand.
I too had a level <7ng/ml back in January and have slowly been recovering from it. While most of my symptoms were on par with a vit D deficiency, I keep having "anxiety" attacks. I don't think they're textbook anxiety attacks but it's more of a feeling of jumping off a building, riding a roller coaster. These attacks were way more intense back when my level was extremely low. In fact, it would wipe me out for the day. I'd feel extremely fatigued afterwards.
While I'm still having these attacks (my level is around 50ng/ml for the past 3 weeks) they're a lot less intense and last a fraction of what they were...but I'm still having them.
I may have something else going on too (need to check my thyroid) but all of my symptoms have improved since getting my levels back up but I'm far from normal.