I advise you to follow the dermatologist's advice and have it off. It is indeed reassuring that the mole hasn't changed, but if the dermatologist thinks it's irregular, it should come off, for peace of mind if nothing else. It will probably turn out to be just a mole, but the quicker you find that out, the happier you'll be.
Please let me know how it turns out.
Best.
Dr. Rockoff