This sounds like a very frustrating problem, as well as a complex one. It is clear that you have difficulty breathing. The rest is not clear. Because the peak flow does not change when you are having these problems, you do not wheeze, and do not cough the first question is the diagnosis. It is possible that this is asthma, but it certainly is not definite. It is probable you are allergic, but it is not clear that this is the cause of the current problem. I would highly recommend a thorough evaluation of the difficulty breathing to get a better idea as to the diagnosis. Then, I would suggest getting control of the symptoms and treating the cause. That is easy to write, but not as easy to do. This is why many people come to National Jewish Medical and Research Center for evaluations, when the obvious reasons are not the cause. It is possible you will need tests to look for other problems such as vocal cord dysfunction, sinus inflammation causing a reflex in the lungs.
Good luck!
Sounds like its time for a 2nd opinion; preferably from a university hospital top rated in pulmonary disease; see US News & World report ratings.
PFT [pulmonary function tests] can help diagnose your lung condition. Chest x-ray and thoracic CT scan might be indicated.
Maybe a treadmill test with EKG and pulmonary monitor. Check your oximeter readings, when you visit the doctor.