Was it an open sore, or was it like a pimple? Herpes looks like an open sore or blister, and can pus. If that's the case, see your doctor ASAP if another sore pops up as Westguy said, they need a fresh sore to test. If it is more like a hard bump, it may be a pimple or infected gland. Good luck.
Sounds like herpes. Painful, burning sores are pretty much considered herpes until proven otherwise. But it also could be an irritation or just a run-of-the-mill vaginal infection.
Definitely get to your doctor and/or gyno. The best way to get this checked is to get a doctor to look at it when the sore is actually there. There's really no reliable test for herpes except visual confirmation and he/she could also tell you if its something else.