YES!!! Which is why there is never a safe time to be having unprotected sex, and never listen when a man says something like that. Use condoms AT ALL TIMES.
A guy's going to say anything to get out of using a condom. If you are not on birth control/not taking it consistently at the same time daily, I would definitely be concerned. I would also be concerned about STDs and the like. Don't believe a guy when he says that. It's likely to be a ploy even if it has any ounce of truth to it. Always use some form of protection.