A really high percentage of adult population got herpes during their childhood, so they can spread it even being virgin when they have sexual relationships for the first time. But there must be contact with the active blisters to have an infection.
If you are concern, you can take him to a clinic to test, and test yourself 3 months after the laste encounter.
All the best.