I had a big problem with vertigo while tapering off Zoloft.
I would definitely get another doctor's opinion if I were in your position. Doctors are quick to give you an anti-depressant now-a-days and I went through a similar situation. I couldn't just stop the Zoloft, i had to taper off and then go on another med. I'd suggest a psychiatrist to help you manage the anxiety and the effects your meds have ...