You'd think the doc would at least give you a CT scan, to look for a blockage. But maybe he or she is right. The human body is more complicated than a car, so you can always get a second opinion.
There's lots of things that can cause your problem, like nerve damage, age, or a blockage, but the olfactories are high up in the sinuses, and if there is something up there it could be very risky to operate, which could be another reason for the doctor's decision.
My experience may have no bearing on yours, but maybe it does. I also had a diminished sense of smell, couldn't smell cat litter box and a lot of things. Then I moved to a humid climate, started teaching, and of course got sick a lot, then chronic sinusitis, then totally lost all smell. In desperation, I developed the Flip-Turn Sinus Flush.
http://www.medhelp.org/user_journals/show/2322
The Flip-Turn Sinus Flush is mildly risky, because you have to bend over to do it, preferably in a shower, but you can also do it outside on soft ground.
A side effect of curing the sinusitis was that now I can smell everything, very well, including the cat who pees in my garage.
As it turns out, you can get a chronic anaerobic infection in your upper sinuses, with few other obvious symptoms. I suspect you may have possibly (not probably) have the problem because of the omnipresent bugspray smell.
If you do try the flip-turn, don't be alarmed by the burning sensation. Even if your olfactories are messed up, saline won't cause any further harm.
Good luck, persistance rules