See your doctor for sure! They will give you alternatives and a safe plan for you to wean off carefully. I agree that SSRI's have worked much better in my case as well. Good Luck
The safest way is through the advice of your doctor in my opinion. How long have you been on it? Medications like these can definitely be hit or miss in my opinion. Wellbutrin has been known to actually increase anxiety in patients. In my experience, the SSRIs like Zoloft and Celexa have worked much better to curb my anxiety, but we are all different...keep us posted!