This is not related to an infection. You should take him to a EyeMD-ophthalmologist.
or if it is more fequent and he seems to kind of be a little disoriented he coudl have absense seizures.
I don't know the answer, but I have seen many peoples doing this, when - I assume - I looked them straight in the eyes. In my opinion, this happens when they feel unconfortable, and that instead of reacting, they make this eye movement (like someone that is blinking his eyes, but it seem to be a deeper "pulsionnal repression" in this specific case). My personnal (non scientific) impression.