It can only hurt you if you do not change them every day and get some weird pelvic infection. A person would have to be pretty nasty to leave it in a long time though - however it has happened. Change the tampon at least once a day and you are fine. The people who I have ever known of left the tampon in for weeks and such. Uck. I myself have always used tampons - for 27 years now with no problem from that. Only infertile from old age now.
I have never worn pads (except for after my dd) I'm a sole tampon user and have been pregnant 4x.