Lisa, it's only been two days. It is perfectly normal to experience such symptoms as you body is just reacting to a new substance in the body. As you build up tolerance for the drug, these side effects should subside. I am not your doctor, but if I were you I would stay on the drug for a few more days/weeks, see how you feel.
Yes you should be perfectly safe in stopping this medication because you've only been taking two pills. You should let your doctor know that you are stopping.
It would probably be OK to do that but you should usually only stop taking medication after consultation with your doctor.
It's possible your doctor could try a different medication. I personally would recommend getting psychotherapy to help address the anxiety.
J