Hi Mic, I just had a 4-5mm stone removed from my ureter. It was not causing me pain, as it settled and quit moving, but I wanted it out. I have read though that stones of any kind can take up to 8 week or more to work their way out. The thing that matters most as you know is the size and lots of water.. Also read 4mm and under usually pass on their own.
Gary In Macomb, MI.
If it was me, I'd go get it taken out. The uncertainty of that stone being there and not knowing when it will cause you severe pain and really mess up any plans you have would be enough for me. I went through that in Spring of 2013 and after three months had stones removed. Good Luck and please let us know how you are doing.