I agree with Judy on this one. Anyone who is that determined to get up so early everyday of their life to work out all the time must be tired at the end of the day. If you're in love with one another, speaking about this shouldn't be such a stink. Try bringing it up in a polite and casual manner and see wha sort of responses arise from that.
Keep us posted on how everything goes, and try to keep your head up right now. I know that's easier said than done, but there's something to be said about communication within a relationship. Believe it or not, having sex even just once a week is a lot to some people here on the forums. This might not necessarily mean that he's not attracted to you, but it could mean a number of things as well. One of those things being, that he is comfortable enough within the relationship the two of you share, that he doesn't feel the need to be on all fours everyday of the week, if that makes sense. My husband and I have known each other all of our lives, for the most part. We don't have to have sex all the time -- but we do maintain a healthy sex relationship where it's unplanned once or twice [or more, depending on how awake or tired we are, haha] a week. In all honesty though, for the most part it IS only once a week. That doesn't mean that we don't love each other though -- it just means that we're truly comfy with each other, so we don't need to show that all the time under the sheets.
My first response as possible infidelity, but not in all cases. It sounds as if he is just tired from a long day of work and gym, but it doesn't excuse weekends. Try it his way for a short trial period and if that doesnt' work for you, it's time to discuss your concerns and how you're starting to feeling neglected in the bedroom. Communication is key in a relationship, so try it his way for trial period and take it from there. Good luck!
Try having sex on his terms; then ask to have sex on your terms.
Perhaps, you can arrive at a compromise that'll satisfy you both.