In Graves disease, the main issues are the Free T3 and Free T4.
Forget the TSH level as that very rarely rises if you still have a thyroid and Graves.
Your Doctor would be concerned about the Free's rising and thyroid storm which can be dangerous and can kill you.
Maybe a small dose of Carbimazole for a few weeks may do the trick...maybe 5mg every 2nd day.
Maybe you could suggest this to your Doctor.
Dont mess with Graves as it is the most hardest of all thyroid issues to treat and can kill you if left untreated.
Not many go into remission...some do but not many.
And once you have Graves, you have it for life.
You have had it for 5 years.....5 years of excessive strain on the heart muscles, excessive strain on the Liver from the anti-thyroid meds.
I would take the advice of your Doctor personally.