Yes, if you got rimmed, you need an anal swab.
Also, an easy exam can tell if you have hemorrhoids. Did you get an exam, or just testing? If you only got testing and no exam, just make an appt with your doctor. Hemorrhoids are really common. I know it's maybe embarrassing for you, but your doctor has seen 1000s of them.
The risk is really low from rimming, so if you want to get get a cream for hemorrhoids and try that before testing, that would be a reasonable choice, too.