The topic of HPV is a complex one. I will try to provide some facts. For addition information on this most common of STDs, I would suggest search for other HPV- and wart-related Q&A on this site, as well seeking addition information on sites such as the American Social health Association web-site (disclosure, Dr. Handsfield and I are both on the Board of ASHA)..
Now, on to HPV. HPV is the most commonly acquired STD. Over 85% of sexually active women will have HPV infection at some time in their lives. Furthermore, nearly 50% of women with a single sex partner have HPV within two years of becoming sexually active. I say this not to frighten you but to put things in perspective and address your posssible downstream concern of a future BF not believeing you (if he doesn't believe you, perhaps he is not the right guy for you) In some HPV will cause genital warts, in others it will not cause warts but may lead to changes in PAP smears. In nearly everyone who gets HPV, warts or otherwise, the infections will resolve by themselves without therapy in 8-10 months. In a very small minority of women, HPV infection can persist and lead to the pre-cancerous lesions that PAP smears detect and which can then be treated. With this as background, on to your specific question:
You are right, HPV is simply not effectively tranmitted by hand to genital contact.
Hope this helps. EWH
thanks so much for the quick, detailed reply!