My son is without his biological mother. He lived with her for the first five years of his life and then one day she gave my husband and I custody. In the last two years she has seen him 3 or 4 times, two times were only a five min. hello and then goodbye. We just tell him when he asks that she loves him and that he has done nothing to make her leave. We also tell him that we will never keep him from her and that when he is old enough we will tell him all the details...so that he is old enough to understand. I hope this helps.
The goal is not to determine how your son will feel - he's entitled to his feelings, and if he does feel sad it's a normal response. All you can do is be truthful, and do so in a careful and sensitive fashion. Try not to go out of your way to criticize his father, even though you have your own thoughts and feelings about his father. Focus on what you know about how you and his father met, for example, what sort of work his father does (or did), things his father liked to do. It's OK to tell your son that his father hasn't come in touch, and that you don't know why that's so, but that some people are not prepared to be parents due to their own personal problems. Invite your son to ask questions, and do your best to reply in a simple, clear, straightforward manner.