I am not sure what you are asking here.
If you just want sex, what's the problem? It's not your friend's decision or ours............it's YOURS.
If you want something meaningful, then this isn't it. He doesn't want you to be there seriously in his life, e.g. when his mother died; he wants you to be available for the booty calls.
What's the "more to the story?" That would be interesting to know.
My observation :
You are BOTH "all about sex" (You: " His penis isn't all that big " AND "I waited all this time (?!) for this smal **** " (?!) You asked not to be judged but I must say that sure strikes me as an incredibly immature thought for You to have !!
You've said nothing here that suggests to me EITHER of You are looking for a 'meaningful' relationship - just sex
a 'meaningful' relationship would be difficult to obtain with such an age difference anyway. You are only TWO years past being a teenager - 14 YEARS past being a teenager for Him !! That's too big an age gap at Your very young age.
You should be looking to meet someone in Your own age group - maybe someone with the same values and standards as You ......just saying