What could cause a significant drop in FEV1 (increase in Lung age) in one year?

I had spirometry done in Feb 2010 on my own request, as I had been getting very short of breath when exercising strenuously - Dr thought i had anxiety.
They results were 'normal' - or more specifically a lung age of 33, (I was 31) which they said was fine.
I had another test done march 2011, again, my request to see if there was a difference, as i was getting short of breath for no reason sometimes (it came and went) - Dr said anxiety, and that i was too young for copd.
I insisted, and the result was a lung age of 39 - and the dr told me he thought i had copd.
I have since had a 3rd one done by a private respiratory specialist for a 2nd opinion, and again got 'normal results' but still lung age of 39. He also did a chest xray. He said I dont have copd, and even with lung age of 39, he said the numbers were still well within those acceptable as normal, and that i should pack up smoking.
However, I have smoked since i can remember - approx age 11 or 12, and dont understand, and nor does the respiratory specialist I have just seen, why i would have done little/no damage up to feb 2010 (19-20 years of smoking) and then in one year taken a sudden rise in lung age. The only thing different from a year ago is i have stopped my regular strenuous exercise - could that really make that much difference?
The specialist has advised I take peak flow meter readings every day for a few weeks (been doing for a couple of days now and getting 650 every time) and stop the smoking.
I have since stopped the cigs, just in case - but would like to know if anyone has any ideas?
My latest results were at age 32, 185cm tall, male, FEV1 4.53 and FVC 5.18 = lung age 39 (a year ago they were fev1 4.59 and fvc 5.01 lung age 33)
Thanks in advance for your ideas!

[allmymarbles]

The age business is not as reliable as you think since performance will naturally vary. In any case, with those readings you do not have COPD.  It could be anxiety, it could be high cortisol, it could be a touch of asthma, it could be lots of things. The next time you get breathy, take an anti-histamine. If that relieves it, your problem is asthma.

[allmymarbles]

Actually what I meant to say, was an allergy.

[Mustard79]

Thanks for the reply :-)

Tried the antihistamine last night, can't say i noticed a difference, but then i did fall asleep fairly soon after taking it!
I do breathe deeper when exercising, but no more than anyone else i know - what bothers me most is when i feel like im not getting enough air when im just watching telly, talking or eating, hence why i did the spiro.
Im having real trouble self interpreting these results though, it seems my fvc has increased, whilst my fev1 has dropped, which gives a low fev1/fvc - maybe im not performing the test properly each time?
Anyway although my results are still 'normal' i cant help worrying about the sharp rise in lung age in just a year, im convinced i'm headed towards copd, i mean if it drops again like that in 1 more year, then it certainly wont be normal next year.
For that reason, i'm still not smoking, and pathetically really missing my bensons whilst sucking on an electronic cigarette!
Thanks for reading :-)