It sounds to me like you have internalized the pain instead of dealing with it. It sounds like a protective mechanism to ward off allowing yourself to emotionally connect on that level again for fear of more hurt.
I have heard it said, that to forgive is not to forget. Somewhere deep down you have done neither. I think it might be time for some counseling as it is not fair to you or him to continue this current mode.
I dont think cheating says as much about you as it does him. He is the one that cheated therefore, it is he that has the problem. Getting to the root of why it happened and dealing from there is necessary in order to move on.
Your hurt that he has been intimate with another woman. It's noble of you to stay or forgive him, but you haven't forgotten and trust has been broken. You do have options and choices. It sounds as if you are in love with him and this is why you did not walk away and not look back after you discovered he cheated. You can't change him or his behavior, but you can change you. The only person you can change is you. If you want to stay in this relationship with the reality that he did cheat on you, then you must forgive him and move forward as is. If he cheated then he was either not ready for a serious, committed relationship and has needs that were not met or it just happened. It's up to you how you want to handle this. By letting it haunt you, it's not going to solve anything and trusting him is going to be an issue. Does he want to be with you? Why did he find the "need" to go else where? How serious is the relationship? There is no excuse for infidelity unless he is just not ready or want to be committed to one woman at this point. Communication is key here, so you need to discuss with him what is it that he wants, needs and what type of relationship does he want with you, exclusive or not. This is fixable, but it's really up to you if you want to stay with a man that is high risk for future infidelities, because you love him or is there something in the relationship that needs to be addressed, such as exclusivity, committment and what need was lacking that he had to go elsewhere. Good luck.