I would believe the ophthalmologist not the optometrist. The optometrists is not a physician, does not do surgery and often not interested in getting you out of glasses. Toric IOLs can routinely correct up to 4 diopters but other methods including laser, incision placement and LRI can be used for larger amounts.
Your optometrist is something of a drag and has limited understanding of cataract surgery. It is true that Medicare and standard insurance only pays for spherical IOLs that do not correct astigmatism. Since astigmatism blurs both distance and near vision it is almost certain that you vision far, intermediate, near will not be acceptable without a toric IOL. The upgrade price is reasonable, the % of patients that need a second procedure to re-align the toric IOL along the axis of astigmatism is less than 1%. I have hundreds of happy patients with toric IOLs. If you would like to make your vision without glasses as good as possible toric is the way to go. Beyond that is multifocal IOLs, those are much more expensive, have problems with night vision and not a few patients are unhappy that they still need glasses for certain things.