There are lots of reasons other than herpes to explain the pimple you describe and I would not worry about herpes unless the number of such lesions increase or other symptoms occur. There are many reasons why HSV is unlikely. These include:
1. You say nothing about your partner but it is statistically more likely that she does not have HSV than that she did.
2. Even if she did, unless she had an active cold sore on her lips, the chances of getting HSV from a single exposure is low. If she had a cold sore, the chances of getting infected from a single exposure is still low but not as low.
3. When people get HSV nearly all develop lesions 4-10 days after exposure so that your pimple appeared at 3 weeks would be atypical and unusual.
4. As I already said, the lesion does not sound particularly suggestive of HSV.
Putting all of these together, my advice would be not to worry. If you cannot help but worry., I'd then suggest you go see a health care provider who can do a culture or PCR test from the lesion. I predict such a test would be negative. There is no role for a blood test for HSV at this time in your case - too early and if it were positive you still would not know that the lesion you noted was HSV.
Hope these comments help. Take care. EWH