Hello~You may not be allergic to either, but have a sensitivity to each one, and when taken together, it is too much for your system to handle. If a person only has "mild" allergies, they won't present themselves until "reinforced" by another similar substance. I think since you have a fairly strongly reaction when combining them, I would just avoid doing it. It is better than being sick.
I agree with SassyLassie .. hope this is helpful