Hello,
It is not safe to take any medicine in pregnancy without your doctor’s guidance. You can take acetaminophen for the headaches. Try to rest, put ice pack, pain medications and avoidance of triggers may help provide relief. If it persists, you may need to have this checked by your doctor for proper evaluation.
It is very difficult to precisely confirm a diagnosis without examination and investigations and the answer is based on the medical information provided. For exact diagnosis, you are requested to consult your doctor. I sincerely hope that helps. Take care and please do keep me posted on how you are doing.
Try to increase your fluid intake. Your fluid needs increase with pregnancy and dehydration can cause dizziness and headachs. Worst case you'll just have to use the lady's room more often. I hope this helps!