I use a hot rag on my head and cold on my neck. I also take warm baths. Mine stem from my neck most of the time so if I use a heating pad it helps too. Always use a moist towel between the heating pad and skin to make sure to keep the muscles hydrated or it'll end up making it worse.
I was the same way for the first fee weeks....headaches all the time. I actually went to see my chiropractor (who has worked on many pregnant women). It worked wonders!
First of all, you should figure out why is your head hurting. Is it because of dehydration, sore neck/back muscles, stress, food....? Tylenol is ok but it just gives a temporary relief. Know the cause and you'll get rid of the headache for longer.
I get migraines and have throughout this whole pregnancy. For me I take very hot showers and let the water hit and run down my head. Sometimes I would have to do this multiple times a day.