More than likely yes, your vision w/ silicone oil will be worse than without the oil, if your retina is attached. However, in many patients vision w/ silicone oil is not too terrible as it is clear.
Yes, there are side effects to oil retention such as papillary block, cataract formation, oil migration to the AC, etc., which is generally why surgeons like to remove oil if possible when the retina is completely stable.
Be sure to talk w/ your doctor about your concerns. I'm sure after already having been through the process a couple of times you are well educated on the matter.
Good luck w/your surgery.