Condom does not roll-on and roll-off during the back and forth movements. Final coverage you saw after the withdrawal, was the "minimum" coverage, the latex was offering during the intercourse.
If the condom covered your penis in a way, you could not see your foreskin exposed, you are fine
As long as top of the penis including urethra and foreskin (in case of uncircumcised males) remains covered, risk is also covered)