I'm going to step out here and say that you trusted your instincts (that feeling of insecurity prior to checking your significant others email/texts), and you discovered that your suspicions were correct. You are not wrong, and you shouldn't feel guilty for trying to uncover the truth and protect yourself.
Let me ask you this...if you were the one who was writing these messages to other men, do you think your partner would find them inappropriate based on content alone? I sense he may be manipulating you a bit. Always trust your instincts when it comes to relationships. What you do about it is entirely up to you. You can confront him, let him know you still aren't satisfied with his answer. Let him know that although it's not a big deal to him, it is a big deal to you. Try to reach a compromise, but don't ignore these red flags.
Consider some counseling therapy, either alone or with your partner, as you believe best for you. Feelings of insecurity can get out of hand and cause many health problems, especially anxiety.
Keep in mind, love and relationships are always "out of control", because someone else is involved in decisions they can and will make that may affect you. Contact a therapist, and have a good old fashioned talk to, I think that might help.
Best of luck
I think any woman would feel as you do. It's interesting that you were feeling insecure and then found these messages to other women. Do YOU know these women? Do you know how he met them? Why would he be conversing with other women behind your back? Even if they were mutual friends of yours, why would he do this? I think you know there's more to this and it's eating away at you. He's not going to admit to anything, so maybe you need to do more investigating to see what else you find. This needs to be put to rest one way or another, or you will never rest. It sounds like you are having some anxiety and depression from this, so you may want to talk to someone for advice on how best to deal with it all. It may be awful for him to be getting blamed, but he has given you reason to be suspicious and you're suffering. You may want to think about marriage counseling so the two of you can work thru this together or at least get to the truth. I feel so bad for you, the not knowing is awful. Seek help in how best to handle your emotions, and I wish you all the best.