I think Valium comes in 5 mg or 10mg, but if, and I'm not a dr but am quite familiar with panic and panic attacks: this should actually help your state.
Valium is an extremely safe medication (in terms of mortality), and you are in no danger whatsoever of dying from the dose you have taken.
It should be safe, but perhaps you should get a weekly med dispenser and put one bit of your prescribed dosage into each day so anytime you have anxiety, you will take the amount you're supposed to.
You know your body best. If you start to notice anything that concerns you, please get yourself to the doctor's. Based on what I know, you should be fine, but listen to your body. It will tell you.