You are totally over reacting. There is no suggestion here that this partner has HSV2. You had protected sex and no symptoms since the contact. The odds of infection are therefore 1 in 50,000 or more in these circumstances.
Also to add, the past two days, I haven't really eaten or drank anything so that could be the reason for the flu.