Welcome to the forum. Thanks for your question.
I can reassure you from two perspectives. First and most important, I doubt the single skin lesion you had was a wart. Second, you really shouldn't be worried even if it was. Genital HPV, with or without warts, should be viewed as a slighly unpleasant inconvenience, rarely an important health threat.
Genital warts usually are multiple; in my nearly 4 decades in the STD business, I have rarely if ever seen someone with an initial outbreak of genital warts that presented with a single small lesion like yours. More likely it was a skin tag or other minor anomaly of the skin.
You also should understand that getting genital HPV is unavoidable; it happens to almost everyone. You clearly have already had at least one HPV infection, regardless of the cause of the skin bump: HPV is virtually the only cause of abnormal pap smears. I'm glad to hear yours cleared up -- that's what usually happens. Most infections never cause symptoms and clear up on their own. The most important step in preventing the rare potentially serious outcomes (cervical and other genital or anal cancers) and inconvenient ones (warts) is to be vaccinated with Gardasil, the vaccine that prevents infection with 4 of the most troublesome types of HPV. Assuming you have not already been vaccinated, I encourage you to do it now. Below is a thread that discusses some aspects of genital HPV and warts in more detail; you can find many others by using this forum's search window or just scanning a bunch of threads.
http://www.medhelp.org/posts/STDs/low-risk-HPV/show/1498003
As for a partner who would take legal action in event of an STD -- well, that's highly inappropriate and unrealistic. He will never successfully avoid HPV, any more than you or any other sexually active person will -- and the odds are he has also already had one or more HPV infections, perhaps including the one that caused your abnomal pap smear. Not to mention the possibility of other asymptomatic STDs in partners who by all appearances are at little or no risk. For those reasons, he would be unlikely to ever be succesful in such legal action. (This forum doesn't give non-STD advice, but a person who would threaten legal action for catching an STD makes me nervous about the health of your relationship.)
I hope this information has been helpful. Best wishes-- HHH, MD