Maybe you misunderstood your surgeon. I think it is standard procedure or treatment if you have cancer cells although you have clear margins the follow up too Lumpectomy with radiation is recommended.
The Oncologist is correct; Radiation is always recommended after lumpectomy, clean margins or not. Some do work thru radiation but it always causes some fatigue and it doesn't stop as soon as radiation is finished. It would depend on what type of work you do and how often your treatments are scheduled. I'm just basing this on what pt.s have told me; no personal experience here. Good Luck .....