Definitely go to the doctor. Lazy eye is often caused by myasthenia gravis, but not usually with pain. It could be caused by an infection in the muscles around your eye though, or several other issues, all of which should be addressed and treated by a doctor.
Wishing you well..
Why haven't you been to the doctor? The pain probably isn't in your temple, it's probably in the muscles surrounding your eye. There is a possibility that you are experiencing a type of lazy eye. If you are, you have probably had it all of your life but it wasn't severe enough to be noticed. You need to see an eye doctor ASAP, in case it's something else, more serious. In any case, it needs to be taken care of soon.