Don't worry, your body will tell you if you can't have more than a few. I occasionally have about four glasses of wine spread out over a whole evening. So not drunk. But I still get a hangover the next day, a definite signal that this is really not o.k. It also has to do with my chronic fatigue. But you might be different. Good luck!
(I had a cva 2 years ago and am reasonably recovered. No overt signs on the outside)