If you have no sores, what are you calling outbreaks? Have you been diagnosed by a doctor? How were you diagnosed, if you were?
10 weeks is a long time from infection to symptoms, but it's not impossible. It's also possible that your husband has it, never had symptoms and gave it to you.
Does your husband, or did the other man, give you oral sex? You can get genital herpes type 1 (ghsv1) from someone who has oral herpes type 1. Even if the other guy gave you ghsv1, and your husband has oral type 1, it would be very unlikely for him to get it genitally.
If you got hsv2 from the other guy, and are having outbreaks and having sex with your husband, he will get it eventually. How will you tell him then?
I am wondering how you were diagnosed with no sores, though.