From what you have described, this can be due an underlying infection causing abscess formation (collection of pus). This can be treated with antibiotics and/or incision of the area to drain the abscess. Other uncommon causes would include Paget's disease of the breast, with an underlying associated breast nodule. I suggest you have this looked at by your doctor to have a more accurate picture. If this is indeed infection, he may give a course of antibiotics.
Wishing you all the best. Regards and God bless.