Talk it over with a counselor. He didn't tell you what he should have told you, which is that he is actively seeking help at not being sexually interested in seeing kids have sex. If someone arouses to the idea of children sexually, this might not be something that will ever change about him. And in fact, it doesn't sound like he is saying he intends to try to change these thoughts, just that he is not proud of having them. In a way, his being ashamed is the (small) price he pays to be able to continue to have the thoughts. Was he asking for your help in seeking therapy, or was he merely checking out your reaction in hopes he could go on like this with your (if not approval) at least your acquiescence?
He apparently was trying to reassure you when he says he would "never make anyone have sex with him nonconsensually." But that is just him telling you he would not rape someone, not him telling you he would never try to act on his attraction to the idea of children having sex.
Hard life or not, however he got there, he is there now, and it's what he decides to do to either rid himself of these desires or to control them so he never does anything to a child, that matters.