Brain scarring and long term memory loss

I have major long term memory loss. I think short term is okay. No drs believe the long term but I barely recognize my own kids (in late 20's) in photos as children. Past that I don't remember doing anything with them. Had first grand mal at 16, nothing since. Have been off and on meds over the yrs because of pregnancies and drs that didn't think I needed it. Had an MRI last week and was told I had scarring. I don't recall ever having any head injury. My epilepsy is in the left frontal lobe. Have been told I had Optic epilepsy. Could the scarring get worse. Can it be the cause of my long term memory loss. I had one decent doc in Roch NY that told me when I was in my 20's that i would have memory problems as I got older but who listens to stuff when your in your 20's. Worried about the scarring though. For some reason, I don't seem to have any friends and have alienated family (they don't know about this)  Thank you