So you probably are getting false positives. Obviously, I can't say for certain, but since your number is going down, and you had the negative supplemental, it's quite likely.
Anything below a 3.5 has a 50/50 chance of being a false positive, and the lower the number, the higher the chance of a false positive.
The blood tests look for antibodies, and to find them, they look for things in the blood with similar molecular weights. Sometimes, it picks up things that aren't antibodies that have similar weights, like normal blood proteins, causing a false positive.
There is a test called the Western Blot. Unfortunately, it's expensive, most doctors don't know about it, don't understand the limitations of the IgG, and unless you can convince one to order it for you, insurance won't cover it.
You can get it through the University of Washington, but a doctor or medical professional has to order it for you. There's a nurse practitioner named Terri Warren who will do that for you.
https://westoverheights.com/getting-a-herpes-western-blot/
All told, it will be several hundred dollars by the time it's done. It wouldn't be unreasonable to just call your tests false positives and call it a day. It's up to you if you need further testing in order to move beyond this.
That wouldn't explain your symptoms, though, if you test negative. Have you tested for other STIs, like gonorrhea and chlamydia? Mycoplasma? Trich? Were your encounters protected?