You are correct when you say "I think I conceived in February," at least, you are correct to think this if your baby came when he was due.
I assume the person who "said you had sex in the middle of December" (who you remember as having had sex with in January), is not the guy you had sex with around February 10? You said you had sex with him "the first month of January 2015," did that mean the first week of January? It would still be a month too early to produce a baby born on November 4 if it was the first week of January.
Have you done a DNA test with the person with whom you had sex at the beginning of February? That might be a way to start to get things cleared up.
If you gave birth to a healthy full term baby on Nov. 4th, it's likely you conceived around Feb. 10.