If this man has already been removed from a previous employeer b/c of his actions, I see no reason that it would shock people if you said you did not wish to do work with him. Obviously you're not the only one who's had a problem with this guy.
I also think you need to seek out some therapy. If you are still having flashbacks you need to find a way to better handle your environment. There is absolutely nothing wrong with seeking out help. Having someone to bounce thoughts and feelings off of in a controled environment can be extremely helpful for victims of abuse. I don't think you are over reacting though. Someone who has been through sexual abuse seem to have a sixth sense when it comes to spotting other "abusers".
Do you have a boss that you can go to and be honest with? If so, I would ask to be relieved from working with this client. I would share about your flashbacks and the problems that you are having from working with this man. I would then begin to seek other employment in a safer environment for you.
I agree with what Margy said.
I would like to add, though, that whether you choose to stay or go depends on what kind of a facility you work in.
If you are in a facility where sexual misbehavior is not expected - like, for example if you are a dental hygienist or physical therapist - this guy needs to go. He can't be passed on to you for your care because he's too offensive for the other worker to tolerate. That doesn't make sense.
If you are working in a facility that in fact DOES usually see socially inappropriate behavior - drug rehab, head injury therapy, for example, that's a different thing and it would be expected that redirecting sexual acting out would be part of the therapy. If this is the kind of field you are in, really, Lynn, I would suggest you find something else because you will be repeatedly exposed to this behavior from various men.
Best wishes. This sounds very difficult.
Thank you for this post it may help others who have been going through similar,I am so sorry this has happened to you, have you ever had any therapy /counselling as I think it could help you.I wouldnt call it over- reacting , reacting more like it, possibly you have to get some help with the thoughts about the guy at work who looks like the original abuser,but in fact has done you no harm, you could leave the job but equally it could be a good thing if you were able to face it and stay , The fact you are going online to look up the past offender shows you need to get past this point so I would definatly suggest some help from a Therapist.Maybe others on our forum will have more suggestions for you...