I think you've already answered this -- when you stopped, the problem stopped.
I have used Afrin for nasal congestion. And to my ignorance, I was actually used it for several months when you were supposed to only take it for 3 days. After stopping for several months, severe anxiety attacks started to occur. I'm thinking now it might've been a rebound effect of Afrin. But I've read somewhere that Afrin is one of OTC medications you should not be taking(as a warning). Might be best to talk to your doctor first and foremost before using it. I wouldn't recommend using it at all. As it caused my body's high blood pressure to shoot up too high, heart raising, paranoia as if i was about to have a heart attack, and feeling pins and needles on parts of my face and hands. It was one of the most horrible feelings I've felt in my life. So as a warning, if anyone has not taken this medication and is thinking about it, please don't or do not take it longer than it says(preferably only for 3 days, that's it!). If your allergies or congestion do not get better, please see a doctor!
I can affirm the same result using affrin and will stop its use immediately.
After consulting with the pharmacist, they suggested using benadryl to cope with the current episode, and then discontinue using the product.