The risk of getting HIV via vaginal sex is really low. That's not to say it doesn't happen, but it's lower than anal sex. Besides, there's no real reason to think that he has HIV.
You can always ask him to test to help ease your mind. Given your high anxiety over HIV, that would be a best practice with this and any future partners prior to having unprotected intercourse.