I lose my erection when I get tired during sex

Im 28 and for the last 4 years I have been having a problem keeping an erection during sex.  I quit smoking and started eating healthy and went to the gym for about 3 months around 4 days a week.  I felt confident to have sex with my girl friend of 5 years and again I lost my erection mid way through sex when I lost my breath from having sex.  :(  I have been to the doctor several times and had a full check up on  blood work done and everything medically is normal apparently but I dont understand why this is happening.    

I ended up losing my 5 and a half year girl friend because of this.  It got to the point where I never wanted to have sex because i lost of confidence in my ability and eventually it ruin our relationship.

If I'm receiving oral or not doing much work it seems to be fine for the most part but this doesn't help my case.
I know its not good but I now gained weight after our break up and started smoking again.  Please tell me what I can do to fix this.  I know i need to quit and I can but I need a plan of action to follow.  Please help..  

One more thing...  I also tried Viagra once and I got really hard and felt confident and while I was doing my thing on her it again I got tired.. lost my breath and boom gone.    I also have A.D.D. (Attention Deficit Disorder)  but I am not on any medication for it.  I also tried going on meds for it thinking it would help me with sex and it didn't.  I have never had this problem before... it just started all of a sudden about 4 years ago. :(
Please help I'm so lost...

[AvaG]

Actually my ex used to have the same exact problem. He was a chain smoker and heavy drinker, though... I knew his dysfunction was due to his health and the fact that he didn't take good care of himself. I know it depressed him, too, that he couldn't keep up with me, so he ended up gaining weight.  He also tried using Viagra but was so afraid of the side effects that I think that inhibited him as well.  I used to think, too, that maybe it was all psychological and he needed to just relax and not stress about performing so much, so I tried everything I could with him but nothing worked. After being together three and a half years and getting engaged, I finally had to leave him because he wouldn't change his lifestyle and chose not to get healthy. So I have to say kudos to you for doing something about it!

My current boyfriend has never been a smoker or drinker and is extremely health conscious, but he still worries about being able to stay hard long enough.  He has bad knees from skateboarding so I always have to be on top which probably helps him not get so tired, too... but he says it helps him if he can ejaculate inside of me because apparently that's what makes him stay harder longer.  Since he's been with me (we've been together six months now) he's been able to stay hard for four to five rounds which he says he's never been able to do before. The only thing is my ex used to come inside me, too, but he was so sensitive, he had to pull out right away after and didn't want to be touched... I still think it was all psychological. Like I really think he needed to just relax and give in to the feeling...  

I don't know if this helped at all, but I hope it gives you some things to consider trying at least.