Lots of things cause men to have issues with erections - stress, medications, illness, anxiety, hormone levels being off, etc., etc.
I would start with an appt with his doctor, and have his hormone levels tested, and a physical. Does he have high or low blood pressure? A heart issue? Is he on any medications that could cause this? If he's losing the erection when he's inside you, maybe he can only maintain it for a certain amount of time? Is there a basic pattern that you all have to sex? Try varying that to see what happens if he enters you sooner.
Ruling out a physical problem is always the first step. If he's fine physically, then look at the psychological causes he might have. Since he's losing the erection when he's inside you, maybe he feels anxiety or pressure to make sure you feel good. Was he a virgin when you got together? Maybe he isn't sure he's "doing it right". Are you reassuring him that sex with him feels great? Do you have an open communication about it?
I wouldn't think it's you. If he can get an erection, that means he's aroused by you. I know it's easy for me to say that, and I might feel the same if I were you, but it really does sound as if he's attracted to you. :)
Don't make him feel badly about this. Just let him know you want to rule out any medical conditions because you love him and want him to be healthy, and erection problems can alert to other, possibly serious health problems.
I agree there can be a lot of reasons why a man has issues maintaining an erection. Big one is stress. Is he depressed or stressed out? The psychological component to sex is huge. Is he taking any new medications? That can impact things sometimes too. I think he should talk to his doctor about this to rule out anything medically related and then go from there. It's not you. It will work out.
Maybe it’s the psychological weight of the big change in your lives? You said you’re newly weds. Getting married is a big deal and can put things in different perspectives.