There won't be any lasting harm if you don't take the prednisone. It is Okay not to take it.
If your blood sugars don't go down back to normal in a few days, do see your doctor.
Occassionally steroids can cause blood sugars to stay elevated for extended periods of time, even after they are stopped.