Hi I take it you mean 4 weeks into a pregnancy, and he fills its safer for you not to have sex, as an orgasm at this time into a pregnancy is not a good thing for you, so I'm sorry to what he is doing is right.
I think the first 12 weeks are the worst for having sex and they say the last 4, but at the end it never stop me and the wife right up the the night before she went in to have it started.
Good Luck
He may just be afraid of hurting the fetus. Most men don't understand our "inner anatomy". Educate him about where the baby is inside you and explain to him that he's not gonna be "bumping into" it.
It may have something to do with a past experience as well. I have a friend whose wife lost her baby two days after they had sex and he still blames himself.