Welcome to the STD forum. I'll try to help.
The bottom line is that the risk of sexual transmission of genital HSV-1 infection is very low, probably especially low in persons like you, with few symptomatic recurrences and taking. Whether it is low enough for infected people to be free of the need to inform their future sex partners is a judgment call -- an ethical issue without a clear right or wong answer. Clearly the risk of transmission for any particular sexual episode is very low in your situation, especially if you indeed use condoms consistently. Personally, I don't have a problem if people in your situation don't inform all sex partners of their genital HSV-1 infections. However, others would disagree with me on this.
As frustrating as the "double standard" may be, it's not terribly surprising, given the psychological factors that influence sex, partner selection, and so on.
Here is another thread from a year ago, which goes into some of these issues in more detail. Perhaps you'll find it helpful:
http://www.medhelp.org/posts/STDs/Recently-diagnosed-with-Genital-Herpes-HSV1/show/969931
Finally, you might consider starting a discussion on the herpes community forum. You'll probably find quite a range of opinions on the obligation (or not) to inform partners.
I hope this helps. Best wishes-- HHH, MD