I assume your doctor described the statins for elevated cholesterol levels in your blood? Otherwise it makes no sense to me.
If the measured levels in your blood are elevated, it might be a good idea to take the statins to avoid the clotting of your arteries.
2 weeks is very short to adapt to the medication. I would at least take them for another 2-3 weeks.
If by then the symptoms are still there, you could stop taking statins for a while to see if your symptoms disappear, to be sure they are from the statins.
After that you could try an other statin. There are several different statins, with different side-effects.