Breathing problems for days after exercise

I've been having some troubling symptoms, and they've been present for several years.  I tend to have trouble catching my breath for a day or two after I exercise, particularly if I run or do any heavy cardio activity.  I've been to the doctor with these symptoms more than once.  They usually measure my oxygen level and listen to my heartbeat, and then tell me that my levels are find and they can't hear anything ususual.  They say it might be some 'mild irritation' in my lungs and hand me an inhaler and steroids.  I've had a chest x-ray and an EKG and they neither has turned up anything.  I'm trying to find out if anyone has had similar problems / symptoms and what they found to be the cause.

I'm 31, 190lbs.  My diet is relatively poor.  I've actually been shying away from exercise in general due to these symptoms.  My extended family has no significant history of heart disease.  Any suggestions?

[cmcneil]

Sorry I forgot to mention a couple important pieces of information and I can't figure out how to edit my post.  I struggled with an anxiety disorder for about a decade.  I've not had any specific anxiety problems for a couple years no and am no longer medicated for it.  I do take amitriptyline 15mg/day to help me sleep.  These problems were present before I started on that medication, and I can't decide whether there have been any changes in the frequency of the breathing issues since then.

[ed34]

Perhaps a lung function test would be a good move. This is where you have to blow into a large machine to test the capacity and pressure. It also monitors the gases exchanged. I had one a couple of years ago and found it quite amusing. They keep shouting "keep going keep going" even when your lungs are empty. A great test though, it will show just how efficient your lungs are.