If your boyfriend has a history of cold sores, then he is already infected with hsv1, which means he is immune or at least highly resistant to a new infection anywhere else on his body. So, I really doubt you would have given him a new infection on his genitals. I would not worry about it.