Hi,
It's really hard to say what could be your best option,since we deal only with breast cancer issues in this Forum.
I see that you also posted on the "Granulomatous-Mastitis Forum" that deals specifically with this condition and you might find it very helpful.I hope that other women there,could give you more information or advice regarding your situation.
Another option that might also help you make the right decision,is to seek a second or even a third opinion from a different Doctors.This way you'll know for sure on how to proceed.
I wish you all the best...