We also had oral sex before we found out we had it. I developed a sore and after a few days it went away. 6 month later I developed a sore same spot only it took longer to heal so I went to dr and swab confirmed hsv2. I've been symptom free 3 years until last month when I had a pretty bad outbreak.
Hsv2 does not like the oral cavity. Hsv2 oral infections are rare. I wouldn't worry about having the infection orally.
Have you ever been tested for hsv1? It is possible to have it and not know it. Many folks get it at a very young age and then don't get cold sores past the primary outbreak. Or have a few outbreaks as a child then it goes dormant.
I wouldn't be at all concerned about passing hsv2 to your little one. Not only is it rare orally it sheds very little and isn't very potent when it does.