when we met up but i sort of made him do it.... He told me later was that right before i came over he was fingering his gf anally and vaginally, and still had some fluid on his hands
LOL it is kinda gross, but you really aren't at any risk here. Assuming his hands were dry, the only thing you are going to get is grossed out.
You just have to do your best to avoid that stuff. I recently got a nasty staph infection, and have no idea how I got it. If you think about all the possibilities, you'd never leave your house again.