First of all, keep in mind that I am unable to diagnose you because I am unable to examine you, this forum is for educational purposes.
The symptoms and story you provide are complex and there is much information that I do not have. From the symptoms/history you describe I suspect you are having Left temporal lobe seizures and symptoms of Left temporal lobe dysfunction that may or may not be related to the seizures.
The Left temporal lobe (In right handed people, and most left handed as well) is the center of language function (along with the inferior frontal lobe. In addition a structure called the hippocampus is connected to the temporal lobe and is responsible for encoding memories. when people have dysfunction of the temporal lobe they can have seizures, memory loss, aphasia (language problems, such as naming, reading, repeating, articulating) and personality changes.
When people have repeated seizures in the temporal lobe this often causes chronic problems (atrophy/gliosis) in the hippocampus and memory problems are common. Another possibility is that the structural lesion you described "cholesterol granuloma" may be pressing on these structures causing the dysfunction.
An routine EEG (electroencephalogram or "brain wave test") is just a 20 minute snapshot of your brain activity. It often requires a multi-hour sleep deprived EEG or an admission for a 3-7 day epilepsy monitoring unit stay to adequately determine how often you are having seizures.
A final possibility is that anti-seizure medications can decrease your alertness and sometimes can cause memory/cognitive slowing, dizziness, double vision and ataxia (drunken walk). You should have a level checked to make sure you are not toxic on your neurontin (although I feel this is unlikely).
Given the symptoms you describe I would recommend more extensive EEG monitoring either the 2 hour sleep deprived or the admission to an epilepsy monitoring unit(EMU) for 3-7 days (I suspect you will need an EMU). From this data they can adjust your medication dose and/or add additional medications to stop your seizures. If you fail multiple medications, then you may be a candidate for epilepsy surgery to remove your seizure focus. Once your epilepsy is under better control, I believe your memory will improve, however sometimes the same pathology that gives rise to the seizures is causing the memory dysfunction and improvement may be limited. I would also suggest formal neuro-psych testing (not just MMPI) to evaluate your visual/verbal memory.
I hope this has been helpful.
The Content on this Site is presented in a summary fashion, and is intended to be used for educational and entertainment purposes only. It is not intended to be and should not be interpreted as medical advice or a diagnosis of any health or fitness problem, condition or disease; or a recommendation for a specific test, doctor, care provider, procedure, treatment plan, product, or course of action. MedHelp is not a medical or healthcare provider and your use of this Site does not create a doctor / patient relationship. We disclaim all responsibility for the professional qualifications and licensing of, and services provided by, any physician or other health providers posting on or otherwise referred to on this Site and/or any Third Party Site. Never disregard the medical advice of your physician or health professional, or delay in seeking such advice, because of something you read on this Site. We offer this Site AS IS and without any warranties. By using this Site you agree to the following Terms and Conditions. If you think you may have a medical emergency, call your physician or 911 immediately.