There is much continuing mystery about HPV and genital warts transmission, reactivation, and such things. Stories like yours are the exception, but nevertheless are quite common.
One important thing, especially in atypical cases like yours, is to be sure the diagnosis of warts is correct. If your husband's doctor is a dermatologist or otherwise is highly experienced in diagnosing and treating genital warts, undoubtedly that is what he has. But many other kinds of skin bumps can look like warts, and misdiagnosis is common. If there is any doubt, and especially if he has not seen a dermatologist, that would be a logical step.
Assuming the diagnosis is correct, it does not necessarily indicate sexual indiscretion on either your part or his. Warts can reappear (or first appear) several years after catching HPV. If you have no other reason to doubt his sexual fidelity, then you should definitely trust him. The important thing here is that warts are a trivial inconvenience, not a serious health risk. And the HPV types that usually cause warts don't cause cancer -- and even if your husband's HPV infection is a high risk type that could risk cancer, the absence of a cervix means you aren't at significant risk.
Bottom line: Make sure the diagnosis is correct; but if it is, don't lose a lot of sleep over this. Accept the mystery as unsolved and don't let an impersonal virus affect your mearriage.
I hope this helps. Best wishes-- HHH, MD
No, you cannot be confident you are truly negative. No test for HPV picks up all infections. However, if you have no visible warts, you have no worry. Most likely, you have been infected with your husband's wart virus strain and your immune system cleared it up. If it hasn't cleared it up, it will do so soon. As long as you have no symptoms, there is no health problem to be concerned about. I'm sure your own doctor will confirm this.
Finally, don't worry about oral sex. Despite very frequent oral exposure to genital HPV infections, oral warts or other significant symptoms are rare. Your husbands warts should have no effect whatsoever on your sex life; any and all kinds of sexual contact with one another are OK to continue without change, and without condoms or other protection.