The obvious explanation ususally is the truth: that is, most likely the red spot you describe was a minor trauma/abrasion from friction. If you had herpes, you would first have a blister-like lesion (or several of them), that over the next several days turned into open sore(s) with scabs, then would heal--all over 1-2 weeks. Subtle "dark spots" probably are not herpes, and herpes never appears as just "dry skin".
Since 20-25% of the population has genital HSV-2 infection, and most of those are asymptomatic, I cannot guarantee that you don't have it, since you have been the partner of a person with genital herpes. But if you do, almost certainly you have asymptomatic herpes plus some other cause of the penile irritation you describe.
It is true that an HSV blood test can give confusing or miseading results. But >90% of the time the test gives accurate results. If you want to pursue that idea, see a health care provider.
Good luck-- HHH, MD