Low Libido even after Androgel

Hello. I was hoping someone could help my with my issue. I'm a 32 years old, 195 lbs. exercise regularly, been married for 5 yrs. I have a very low libido and low sex drive. It started when I was in my early twenties. my libido would come and go gradually but it was ok,  all I had to do was wait a few weeks and my libido would come back strong for a few weeks. but during the last 3 yrs. it takes forever (sometimes months) for it to return and when it does, it's only for a couple of days. I wait about a week or two to have the urges but even then, it's a weak erection. I don't understand, I get morning erection so I know is not impotense. I went to my doctor since I had fatigue and wasn't sleeping well either. The blood work came back and  found that my testosterone levels were low, he said 40 when normal should be over 100. He put me on Androge and prasterone.  it literally changed my life. I started sleeping better, fatigue, bad mood went away. except for my low libido. He gave me Cialis wich is ok for enforcing my erection but it does little or nothing for my libido. He also told me to take it easy, cause it could be life stress. but the truth is that I don't have any stress in my life or at work. I'm too young to be having this kind of problems. My wife don't say much but I know she's getting frustrated.  I don't know what to do know... so please help. any suggestions would be greatly appreciated