No. You will not experience any withdrawal effects from discountinuing the Zoloft if you have only taken it for 3 days.
Also, keep in mind that it usually takes 2-6 weeks for the full therapeutic effects of Zoloft to take effect (shorter time for anxiety and longer time for depression symptoms to subside) and many people have unpleasant side-effects for the first week of taking Zoloft. These usually subside after this time. You may want to think about whether the potential benefits of taking Zoloft outweigh the side-effects you currently experience that could quickly pass when your body adjusts to the SSRI.
How long/often have you been taking Xanax? Are you discontinuing that also? Xanax is only indicated for stopping a panic attack in its tracks and should NOT be used on a regular basis as it is HIGHLY addictive and rapidly loses its therapeutic effects (more and more is required to obtain the same effects). Klonopin is a much better drug for controlling anxiety long term and has very few side-effects. Klonopin quickly reaches a steady-state concentration and is an excellent maintenace drug. Many people on this site swear by it. Look through these boards and see if this is something that may be of benefit to you and if so, you can discuss the option w/ your doctor.
Im sorry I have been on the genric zanax for about two weeks now and it only been lately that I started taking the zoloft that my anxiety has heighted.
My doctor gave 1 mg pills and I cut them in half and take them. I also have been attending CBT courses which seem to be helping alot that why i kinda dont want to take the zoloft and ween myself off the genric zanax.