You may need to try a different type of goggle - it could be putting pressure on the eyesocket and causing temporary double vision, or it could have nothing to do with the goggles at all and you may just have a slight tendency toward double vison only brought on when swimming. Try another, larger type goggles and see if it helps.
MJK MD
I too am a swimmer but have never experienced this. My guess is it is related to the fit of the googles and the effect on the eye muscles. What goggles are you using? Apart from that, there is some evidence that goggles can mess with eye pressure but I personally think the data is unconvincing right now.