Hi,
Anytime you notice a change in your discharge that seems abnormal or odd to you, you should consider making an appointment for a pelvic exam w/ a gynecologist. Yellow or green discharge is usually indicative of a vaginal infection (and sometimes possibly STD's). Since you do use condoms and don't believe he's a risk, it's possibly a minor infection (yeast or BV) that can usually be easily treated by your doctor. Also, sometimes condoms can cause allergic reactions in some women. Unless you've already done so, try using non-latex, or hypoallergenic condoms. If it continues to worry you, go have it checked out by a doctor. Take Care :)