Get a new primary care physician recommended by the neurologist Google "Lamictal" and read about your mood. Keppra and many other seizure meds make all of us a teensy bit on edge. You cannot prove that your original doc was irresponsible, unless you have a witness to him hearing about your sister, AND failing to proceed with reckless disregard for your safety AND the safety of others. Yes, I used to be a lawyer. And you did not get hurt in any of the car crashes. And you continue to drive, when epilepsy is a real, genetic, probability.
I am an epileptic. I was diagnosed just last year, on this date. All of my prior seizures happened when I was NOT in an MRI. Seizure activity only shows up, or can be diagnosed, when the actual seizure is happening. 16 years of seizing and no one knew what was happening. We live in a small community and every time my hub found me post seizure, he called 911 and the little hospital did not have a neurologist. Finally, hub demanded that they take me to U.C. Davis, a hospital that is a teaching facility. While there, they put electrodes on my head and managed to see 300 seizures in one half of one hour. Now I take seizure meds and do not seize. But that driving business, YOU know you should not be doing it. If you kill someone, that is on YOUR conscience.
Well if you don't want to drive no one can make you. However. it does sound like something is going on with you body. You know your body best. In addition you might also want an EEG.
If you feel your doctor isn't listening to you, you have the right to find another one. Telling you not to worry is not a good thing. You deserve to find out what is going on with you body. Keep fighting that battle, and keep us updated.