I am assuming your exposure was sexual intercourse. Was it protected or unprotected?
If someone had a primary outbreak, they usually develop symptoms 2-10 days after the encounter. Sometimes, more rare, it is up to 20 days.
What did your doctor say about your symptoms? Redness and a raw feeling could be bacterial and need antibiotics. Those symptoms are not exclusive to hsv.
If these symptoms get worse or you notice any sores, get into your doctor immediately. The IGM test might catch the very beginning of an infection but it’s a terrible test for diagnosing adults.