Sperm really needs a way to get to an egg. And because there was no contact or *** it would have been really difficult for sperm to find its way. So try not to be too worried. The thing you need to keep in mind about emergency contraceptives are they are a large dose of hormones, shouldn't be taken very often, and can have some side effects. So if you're really freaking out, I would way up the likelihood sperm found its way vs. the amount of medication in the pill.
My advice to you is maybe start taking the pill or another kind of regular contraception. If you're considering becoming sexually active, even if you will use condoms, it doesn't hurt to be extra cautious. Also that way if a condom does break you don't have to be so worried. It also doesn't hurt to keep a morning after pill on hand, just in case. But like I said be careful how you use it.