The base of asthma is inflammation and an abnormal immune response (in allergic asthma), so even if you do not take meds - there is a base level of inflammation that could be bad enough to cause fatigue and other symptoms if you are sensitive to your body.
Try getting enough and good quality sleep, eat healthy , exercise and lower your stress. A good lifestyle will lower the inflammation and should decrease the fatigue.
You should talk to your doctor though about doing a lung function test - this will tell you if you SHOULD be taking inhalers and definitely symptomatic asthma causes fatigue and low energy. Hope that helps!
If you're not getting any air to the brain, of course you will be tired, iv had it for 55 yrs and I take 1 puff of Advair 250/50 and feel fine...The more oxygen your brain gets, the more you will feel better. See a Family Doctor.