Polycythemia as you may known is an increase in the number of red blood cells mainly. It is though the overproduction of the RBC's is due to chronically low levels of oxgyen or over exciatably of the bone marrow where RBC's are produced. The classic symptom of headaches in polcythemia is associated with a couple of possible causes: hypertension, blood clots or increased intracranial pressure. If you current doctor did not associate this findings with the condition I would suggest you seek alternate advice.
Regards
Dr Foot
Have you had scans to rule out likely causes? I'm particularly thinking of an MRI and MRA of the neck and head. I had headaches for years before these tests were done and they identified some contributing factors.
You mention waking up with the headaches. May be unrelated, but that can be a symptom of sleep apnea. If you have any other sleep issues or are sleepy during the day, a visit to a sleep doctor may be in order.