Make sure you're staying hydrated, lots and lots of water, dehydration causes headaches. It could just be the hormones too! If you need to take some Tylenol its safe to do so, if you are worried double check with your doctor. just take the lowest dose recommended.
I was dealing with this in the beginning of my pregnancy. Headaches everyday. My doctor told me tylenol was okay to take. No more than 500mg but I take 375mg