bronchodilators and caffeine in tea are both stimulants and cause a rise in heart rate - when you combine them you may get even more of a response (called syngery) then you would see from either of them alone.
You could try drinking herbal teas that are caffeine free and this help stop that affect.
Maybe for you tea is an allergen. Are you taking this - https://en.wikipedia.org/wiki/Salbutamol ?