It's extremely unlikely.
Although technically only one sperm fertilizes an egg, sperm is ejaculated with a fluid that neutralizes vaginal "chemistry" and allows the sperm to survive and swim.
A few lonely little sperm in there, with no ejaculate, will die.
I posted a minute ago and it didn't "stick" somehow. :( Sorry if this turns into a double post.
Pregnancy in this case is extremely unlikely. Although technically it takes only one sperm to fertilize an egg, sperm need to be surrounded by a lot of ejaculate from the man to neutralize vaginal chemistry and give the sperm a medium to swim around in safely. A few lonely little sperm with no ejaculate will die pretty quickly.