Only 10% or all cancers can be traced to a genetic familial linkage, and so would not readily translate to your children and grandchildren acquiring the risk of cancer. Moreover, this risk is studies in patients with classical breast cancers such as invasive ductal or lobular cancers. Phylloides tumors are in a different category, and the mode of genetic inheritance of this is all the more unclear.
I would just suggest that your children and grand children undergo routine mammogram screenings (age 40 or earlier) to detect early cancers if ever.
Regards and God bless.