I went off effexor xr years ago because it was raising my blood pressure and it wasn't easy. I turned into a very mean person for the two weeks I was weaning off of it. It was kind of scary because I got kind of violent and if my sister didn't know why she wouldn't have understood. Any way, my doc told me how to taper off by reducing my dose and using my clonazepam for anxiety feelings during that time. I just helps if you have support at home while doing this because it is definately uncomfortable.
When you are weaning off of Effexor XR just ensure you are doing under the supervision of a doctor. Never stop or adjust your dosage on your own. The one complaint that I have with these medications is the same issue as yours. It does tend to drive down the sex drive, but weighing the benefits vs. the side effects, would you say your relationship is better off before you were on it or for the past 6 years? I have been on Zoloft and my doc had said the same thing about trying Lexapro if I wanted to, but sometimes it is trial and error when it comes to these medications so I decided to stick with the Zoloft.
It really is a personal choice with these, in my opinion. I always look at fairly simply; Am I better of with it than I was without it? Sure there are upsides and downsides to any decision, but as long as I see the long term benefit, I am happy with it. Let us know how it goes!