If you have any doubts purchase an extra albuterol sulfate inhaler and have him keep him with it if he is at kindergarten or school. A single use of the inhaler (if used inappropriately) is not going to cause any life-threatening problems, and if he does develop an asthmatic attack, it will keep him alive. Get youself a good stethascope and learn how to auscultate his lung fields anterior and posterior, four quadrants. Listen for wheezes, rales and ronchi. You can easily learn to do this as well as a physician.
Has your son shown any other signs of asthma? E.g. recurring wheezing and coughing over a long period of time? Five years old should be about old enough to test if you want to get him checked out. My younger sis has had asthma since she was about your son's age.
Best of luck!
If he has trouble breathing it may not be asthma. It could, for instance, be a sinus problem. Ir could be an allergy.
5 years old might still be too young for peak flow testing but your doctor could always try a trial of medication to see if it helps. I would contact them and ask specifically WHY he doesn't think he has asthma and go from there. If the answer makes sense then go with it but there should still be something that the doctor can give you that will help.
If you don't agree with what the doctor said then it's time to see another doctor if your gut still thinks something is wrong.