Anxiety and depression long standing

I have had agrophobia for most of my adult life.  I am able to control it now with the help of Effexor and  vallium.  I am 55, and when I started taking it, it was considered the wonder-drug. diazapam started to get a bad name and I became more dependant on it.  I am fairly controlled now, with the help of my GP but although I can't imagine giving it up, I'd love to be drug free.  I am scared that I will go back to the person who sat at home all day and spoilt everyones life.  I have a very supportive husband.