You have received good advice already, and take note that Effexor does have a stimulating effect, and most anxiety and depression is a mixture of both, so Zoloft or some other medication may be your answer...as for memory, it's hard to diagnose from this information because it could be from the heightened anxiety..did you try therapy along with medications..if not, you can try the self help therapy I have pioneered at www.myvirtualshrink.com.
Different meds will affect you differently. That's just how it is. Also, because Effexor also works on norepinephrine receptors it can be very stimulating; it's more of an antidepressant than an anti-anxiety med, though many have found help from it for anxiety. It may just not be your med, whereas Zoloft was. As for the sexual thing, that's pretty common with all the snris and ssris -- it can happen with Effexor just as easily as with Zoloft for some people, so there you go. If you decide to quit, please do it slowly and in consultation with your psychiatrist.
I'm hopeless with medications so are not able to offer any constructive advice. Sorry!
It could be that you may need to increase your medication -not sure.
I'm not sure about the memory issue either. I think the doctor said that memory can sometimes be the result of three d's -one being disinterest.
Have you looked at psychotherapy for helping you to address your anxiety issues?