You had no risk so it doesn't matter.
Zero risk. Shaking someone's hand, even if they or you or both of you have cuts, scrapes, bruises, scratches or any other kind of wounds do NOT spread HIV. HIV can only infect inside the body, so if you did not share needles with him for IV drug use and you did not have unprotected anal sex with him then you had 0 risk.