I went through this with after I had my daughter. And I only pumped for three days. It lasted for almost two years. And even to this day (she is three) I will still have some leakage every now and than. After several talks with my doctors and nurses I have stopped worrying about it. Just talk with your doctor to have your mind put to ease.
I had this problem even 5 years after having my child and only breastfeeding for a couple of days. I went to several doctors and both my primary and gyn say that its normal. Some women get this just by having stimulation to the breast area. I have to tell you that it stopped when my daughter was about 10 years old. Just ask your doctor about it but I dont think its anything severe