Welcome to the STD forum
Late recurrences of genital warts are not especially common, but they occur from time to time. However, if you are sexually active, perhaps you have a new infection, not reappearance of the old one. Two strains cause 90% of genital warts, HPV-6 and HPV-11; you might have had one of these 2 years ago and the other one now. Or one infection or the other was caused by some other HPV type.
I see no need to go back to inform past partners about your new diagnosis. They may not have been at risk; even if they were, the wart-causing HPV strains are not generally a serious health threat; if they were infected, they probably would have developed warts by now, so it's too late anyway; and as noted above, you don't necessarily still have the same original HPV infection anyway. In my view, any potential benefit of telling them is outweighed by "unnecessary stress and fear...about something which is actually so common" (your words, and I couldn't have said it better myself).
You and your partner can assume he is already infected with the HPV strain causing your current warts; or he has had it and it's now gone. Depending on when your relationship started, he could even the be source of your current warts. In any case, there is no reason to change your sexual practices with him, He will not catch your infection again; people do not "ping pong" HPV infections back and forth. If your partner develops visible warts, he should see a professional and have them treated. But otherwise, this is not something to lose sleep over, and for sure you shouldn't let it jeopardize your relationship. Warts are a minor inconvenience, not a serious health threat.
Congratulations on getting Gardasil. It covers 4 common types of HPV and is highly effective, if you haven't yet been infected with them.
Here are three other threads that discuss partner notification for HPV infection. You may find the discusions useful.
http://www.medhelp.org/posts/show/763292
http://www.medhelp.org/posts/show/763984
http://www.medhelp.org/posts/show/761416
Best wishes---- HHH, MD