Is he taking ANY kind of meds that would cause these side effects? Or maybe he needs to see a doctor. If he is telling the truth and is totally attracted to you, (and I find it hard to believe that he would risk the humiliation willingly), it would appear to be a possible physical thing. Or it could be stress.... Before assuming there is something wrong in your relationship, rule out everything else first. And just to let you know, once this happens it causes major anxiety within the fellow and that alone can cause a repeat performance, so try to not make to much of it with him other than the health side of things.
I would believe he probably was nervous...this happened w my bf and I the first time we had sex. He admitted that he was so excited to have sex w me that it got the best of him & he ended up being nervous instead. So, don't worry about it to much.
Or maybe he could just be nervous since this is the first few times you two have ever had sex....
These are worst case scenarios...
There could be a few explanations for this situation, physical or psychological. It is too hard to say for sure because without knowing the person more it is mostly just speculation but here are a few explanations: He masturbated earlier in the night or day. He is turned off by you, he has been experiencing other sexual encounters which satisfy his fetishes and true desires( therefore what you have to offer isn`t as good), he is feeling guilty about something. OR there could be some physical problem , or a host of other reasons.
Are there some things sexually which you do not do that he likes. you should talk it over , there is a definite answer that he is probably not telling you.