To be honest I can see your anxiety over this, but in retrospect the valve has done great over the last 10 years. Personally I think 2 or 3 yearly checkups would now be better suited.
I would find another cardiologist. The surgeon himself told you that you needed a yearly ultrasound. If the surgeon had felt sure that the repair was good for a lifetime, he would not have told you that. And this non-surgeon thinks he knows better? I'm sorry, but I would not be at all comfortable with ignoring the surgeon's recommendation. You're the one who is at risk, not the cardiologist, so IMO you need to find a cardiologist who respects both you and your other doctors. Good luck.