If he used a condom for oral, it's most likely he was smart enough to use a condom for anal sex.
If he did not penetrate you with or without a condom, you had no risk for HIV.
Oral sex is not a risk for HIV and he used a condom so there was no risk for STD's.
You do not have HIV but if your anxiety is affecting your life, then by all means test for your peace of mind.
In the future, make it your personal rule.....no condom, no sex. Safe or sorry is your choice.