Hello,
It can be a boil or an abscess. Apply warm compresses,Neosporin ointment and calamine lotion and avoid messing with it.You can take anti-inflammatory pain killers like ibuprofen or acetaminophen. If the lump persists then it will be best to get it evaluated from a gynecologist as you may need oral antibiotics and incision and drainage.
It is very difficult to precisely confirm a diagnosis without examination and investigations and the answer is based on the medical information provided. For exact diagnosis, you are requested to consult your doctor. I sincerely hope that helps. Take care and please do keep me posted on how you are doing.
Yes, you need to go to the doctor and have this treated. You also need to ask why they are forming. Be sure you wash the area front to back every day using a gentle soap such as unscented Dove.