First - I'm so sorry you have had such a hard time conceiving. infertility is horrible and isolating - I know - I tried for 7 yrs.
Nipple discharge unrelated to fertility (hormones may be involved but that would be ruled out simple blood test). I'm going through this myself. ... from what I've read -. could be benign 'wart' type growth, or infection - either way check it out. I have same issue and go for mamogram tomorrow.
For the fertility side - I'm assuming you've had full di8agnostic and your partner has had all the tests not just for count but motility and 'shape' of sperm. In my case I did 2 cycles of invitro with icsi - first one failed. 2nd one I got my beautiful daughter who is turning 6 soon. Had my son just under 2 yrs later. Hang in there. there are some great support ntworks for fertility if you are not already on them.
I was one of the lucky ones and had success. I now have 2 kids 2 yrs apart. (second was natural miracle 9 yrs after ttc )
it probably is nothing - but see a doctor asap to make sure - nipple discharge can sometimes be a symptom of a malignancy in the breast... see a doctor to rule it out.
It's probably nothing but I just know that when ever I go for a pap smear they check for nipple discharge every year. There must be some reason that they do that. I would not wait. Make an appointment on Monday just to be sure that nothing is going on.