Did the dr mention Benign paroxysmal positional vertigo BPPV? my husband has it and it sounds like what you're describing.
Sometimes he gets migraine type headaches with vertigo, nausea and just feeling crappy. It started one week after we had the flu about 5 years ago.
Might want to ask if that's what yours is and if some exercises would work in your case. His just went away on its own after a few years but no clue why. I think it's because he got his ears cleaned at the dr and no buildup left.
There's info on youtube about it, something about moving the particles in your inner ear, google and see if that's what yours sounds like.