Hi:-) I don't think it's your ex's b/c you say you had unprotected sex with him on Nov 4th. If your last period started on Nov 12th, you would've ovulated on or around Nov 26th. Usually ovulation occurs 12-14days after that so you would've conceived around the dates that you ovulated. I hope this helps:-)
Oh also, most people ovulate 14 days after the first day of you last period, which you said was 12th which means 14 days later then or around, you ovulated
You said you had you period after you slept with your ex, if you had gotten pregnant by your ex, you would not have had a period... Also you will only be 5 weeks, all being you have 28 day cycle, which it sounds like you do, you would of ovulated on or around 26th, which is by my calculations when you also had intercourse, making you just over 5 weeks now by a few days.. Hope this helps!