Hi, it sounds like it could be a blocked gland. I suffer from these and when I first noticed one thought I had warts but the doctor told me they are blocked "Bartholamew" glands, I could have the name spelt incorrectly. I have them regularly and they can get larger. What I do is sqeeze them out in the shower - it can be painfull to do as they are small and you have to sqeeze hard and get the spot between your finger tips and the skin is thin. They are similar to a white or blackhead and getting them out is just as hard, but they are nothing to worry about and cant be caught or passed on. Having said that do go and have a thorough check at your doctor.
Herpes sores aren't usually hard and it would probably hurt a considerable amount.
You don't mention any recent sexual exposure and even say you've tested negative for HSV-2.
My guess is that it's something else.